| |
ARES/RACES HTML/JavaScript Membership Database
Philip Karas, KE3FL
If you are joining us after having read the article in Worldradio you
can skip down to Figure 1 and pick up from there
with the detailed explanations and examples of different database searches.
Otherwise, you should start right here.
ARES and RACES have a need for membership databases. I've seen databases
maintained in all sorts of programs, from straight ASCII files to spreadsheet
programs and full-blown database management systems (DBMS). All have their
advantages and disadvantages. I've worked on and developed a number of
different DBMS, from simple ASCII files to relational DBMSs such as ORACLE.
I designed the database, developed, and wrote a C-based user interface
for the National Corrosion Database (NCD-developed in ORACLE and using
their QSL-C tools) when I worked for NACE International in conjunction
with the NACE-NIST Corrosion Data Center.
I have been rather disappointed with the cumbersome nature and size of
RDBMSs and I have come to the conclusion that many databases need not
be in such systems. The NCD was not one of those. When I was asked to
develop an ARES/RACES Membership Database (ARMD), around 1994, I felt
that ease of use and completely free software were the main issues, free
software being the most important.
In 1983/4 I developed a search engine and a design to use "standard"
ASCII files as database files. When the ASCII file is arranged in a pre-determined
way, this search engine could return all, or part of, the data in a database
record when given the search criteria. At that time, it was a DOS command-line-based
program. I found that in 1994, many people did not like DOS command-line-based
programs any longer. So I wrote a C-base menu program to help with the
user interface but it never was used very much. While all members of our
local ARES/RACES organization received a free copy of the DB and the software,
no one, other than myself, really knew the power of, or even how to use
the "Pfind" (c) search engine. This was not for want of offering classes
on the system either.
When I started learning Perl, Java, and JavaScript in early 2001, I thought
that only Perl and Java could be used to host a database and it would
have to reside on a server. Needless to say, we did not want our database
on a server. It would do no good to be there during an emergency that
caused communication lines to fail. Keeping it floppy-disk size (1.4 Meg)
was important for distribution and, again, using some kind of free software
was important.
While playing with the new level 1 DOM (Document Object Module) and DHTML
early in 2002, I made an intuitive leap from being able to read an HTML
file's innerHTML to being able to read any HTML file's innerHTML. Once
that leap was made, it became a matter of experimentation to come up with
the best form of the "HTML" file to be read so that we could use it as
a database. Now, I could see a way of using my "older" Pfind technology,
re-written in JavaScript and HTML.
The form I've selected works rather well. I've tested it out using our
own CARET membership database as well as two other files that contain
QST table of contents-type information for 1993 - 1999, and 2000 - present.
All files load quickly when they are on a client's computer using one
of the "newer" DOM level 1 browsers, IE 5.0, NS 6.0. (This technology
can be rather slow when used over the Internet. I once had to wait 15
minutes for the data from the 150K QST database to download to my computer
so I could search it. But then, this delay would be present no matter
how I wrote the HTML database file, either for use in a JavaScript Pfind
system, or as a straight HTML file. The Internet was just very slow the
day I was testing.)
The really big advantage of this "new" system is that it uses something
everyone knows how to use: an Internet browser. It also uses my old-standard
Pfind ASCII database file. While I could have designed a fully-blown database
file structure and used classes and methods to operate on the data, I
felt that I simply did not have the time for such an endeavor. Also, for
the kind of information we want to save and retrieve, the Pfind structure
still works very well. Simply because it is almost 20 years from its first
use does not mean that using an "old" idea is out. After all, the data
has not changed very much. I found that I was able to put in HTML anchors
to look up people by call sign and I was also able to put in the HTML
code needed to show photos of members, all in a Pfind-type ASCII database
file.
I implemented a simple version of the Pfind key-word searching feature
and I allow the returning of the found data in thee ways. The program
can return the full database record, just the names and address lines,
or just the line of interest. All of these returns have theirs uses. (see
figure 1)
Figure 1
The Simple Key-word Search:
By "simple," I mean that the present key-word search is not identical
to that implemented in the original Pfind program. While we do not need
to understand the difference between the original and the JavaScript implementation,
it is important to understand the kind of key-word search that is implemented
in the HTML/JavaScript program. In this program, the key-word search takes
all words entered and places them into an array of words to be found in
the record field. The present implementation does not look for correct
order of the words found. It only looks for each word and, if they are
all found someplace in the record field, it is considered a match. They
do not have to be on the same "line" nor do they have to be in the order
entered.
This kind of search has its limitations, but for the most part it will
not matter, for this kind of data. An example might help in understanding
the limits of this kind of a search. Let's say we want to find all people
who have 80 meter CW capability, so we search on "cw 80" and this will
match all records that have lines that look like: (see figures 2 - 4)
C: cw 160, 80, 40, 30, 20
But, and here's the catch, it will also match the record that has the
following line in it:
I: Interests: I'm 80 years old and love hiking and doing 20 meter cw.
Figure 2
Figure 3
Figure 4
Neither order of the words nor context are considered, so both lines
will cause a match and, in fact, the words do not even have to be on the
same line to cause a match. I will simply say that this was not the case
in the original Pfind program written in C, but it was far easier to do
things this way using JavaScript than to implement the single-line, order-important
key-word search used in the original Pfind program.
I thought long and hard before I settled on this type of limited key-word
search. I also believe that if I can see no real difficulties I will first
try it and see if it works. If it does, I go with it. So, I went with
the easy way to save time and I have yet to find a problem, but you should
know that the possibility for the return of some "strange" records does
exist.
Types of Data Returns:
One can understand returning the full data record; after all, everything
is there: Name, address, interests, e-mail address, equipment, etc. (As
shown in figures 3 and 4)
Returning just the address lines means we can "cut" and then "paste"
these lines into a word-processing program to print out address labels
if needed. (see Figure 5)
Figure 5
And last, returning just the line of interest can return things like
all the e-mail address lines so that one can form a membership e-mail
list. (see Figure 6 & 7)
Figure 6
Figure 7
Bottom Line:
There are advantages and disadvantages to using this type of "database"
system. One advantage, as stated before, is that the client will know
how to use the main program since almost everyone has an Internet browser
on their computer. I took a poll of the members at a recent club meeting.
Of all of those who had and used computers, the vast majority of our membership,
I found that all of them had one of the newer DOM level 1 browsers on
their computers. In fact, at that time, I was the only one who did not,
since I was still using NS 4.75 on my laptop computer. (I have since upgraded
to IE 5.50, the best I could do on my slower Windows 95 Pentium.)
The main disadvantage is that every now and then a new version of a browser
comes out that is partly brain-dead, meaning that it no longer recognizes
an old standard JavaScript method. This has happened recently with NS
6.0 - 6.21 but, fortunately, the present problem was not with innerHTML.
This disadvantage will probably only be due to a "bug" in the new implementation
of JavaScript and should be able to be fixed with either a patch or an
update.
Whether or not the selection of an Internet browser and JavaScript as
the basis of our ARES/RACES database is a good decision, remains to be
seen.
If you would like to play with a JavaScript Pfind search engine, one
is available on my web site to search past QST article titles located
at: http://www.qsl.net/ke3fl. It's near the bottom of the page. Look for
"qst."
73 de KE3FL
Phil Karras
AEC Carroll County MD
ORS, OES, VE,
Life Member
MA Physics
Sr. Software Consultant
Syntek Systems Corp.
Appendix A - Pfind Database file structure:
<!-- CARET DataBase File -->
*<a name="KE3FL"></a>
Philip Karras/KE3FL
3305 Hampton Court
Mt Airy MD 21771-7201
Z: 21771-7201 Philip Karras/KE3FL
C: Carroll
M: membership good to 4/1/03
I: <a onClick="document.location='ke3fl.jpg';" onMouseOver="Click();" onMouseOut="Base();">
I: <img src="ke3fl.jpg" width=75 height=91 alt="ke3fl.jpg">
I: </a>
S: ------------------------------------------------------------------------
H: 301-831-5073 (145.41, auto: 55)
W: 301-208-6660 X32
@: ke3fl@arrl.net
T: Ticket Extra, 1992
S: ------------------------------------------------------------------------
A: ARRL Appointments ORS, OES, AEC, VE
A: ARES Equipment 60 AHr, 30AHr batteries
E: Emergency Antenna 160, 80, 40, 30, 20, 17, 15, 12, 10, 2, , 445,
E: Emergency Field 160, 80, 40, 30, 20, 17, 15, 12, 10, 2, , 445,
E: Emergency Power 160, 80, 40, 30, 20, 17, 15, 12, 10, 2, , 445
E: Emergency Power 14 days any one freq.
O: Other Emergency First aid, CPR, water safety (certs not current)
O: Other Emergency German language
C: CW 160, 80, 40, 30, 20, 17, 15, 12, 10
F: FM 2, , 445,
S: SSB 160, 80, 40, 30, 20, 17, 15, 12, 10
M: Mobile 40, 20, 10, 2, , 445,
P: Packet 2, , 445,
I: Interests- Building & testing antennas/repair electronics
I: Interests- (4HT, 4mobile) CB, and (4) FRS Radios
I: Interests- QRP building, have: 20, 30, 40, & 80 meter rigs
S: ------------------------------------------------------------------------
C: Call-UP:
S: ------------------------------------------------------------------------
*<a name="AB3CDE"></a>
Harold McOrt/AB3CDE
124 Binary Way
Westminster MD 21157
Z: 21157 Harold McOrt/AB3CDE
C: Carroll
M: membership good to 4/1/03
S: ------------------------------------------------------------------------
H: 301-123-4567
W: retired
@: ab3cde@arrl.net
T: Ticket Extra, 1945
S: ------------------------------------------------------------------------
A: ARRL Appointments ORS, OES, VE
A: ARES Equipment 2 - 30 AHr batteries
E: Emergency Antenna 80, 40, 20, 15, 10, 2, 445
E: Emergency Field 80, 40, 20, 15, 10, 2, 445
E: Emergency Power 80, 40, 20, 15, 10, 2, 445
E: Emergency Power ? 14 days any one freq.
C: CW 40, 30, 20, , 15, 10
F: FM 10, 2, 445,
S: SSB 80, 40, 20
M: Mobile 10, 2, 445,
I: Interests- I'm 80 years old and I enjoy 40 meter cw!
S: ------------------------------------------------------------------------
C: Call-UP:
S: ------------------------------------------------------------------------
*
<!--endOfList...0
empty
*<a name="YourCallSign">
Your Address
Goes Here
etc
Z: (enter Zip-code and Name/call)
C: Carroll
M: membership good to 4/1/0?
S: ------------------------------------------------------------------------
H: (145.41, auto: XX)
W:
@: # (e-mail address) (***** empty ***** bad *****)
T: Ticket Novice, Technician, General, Advanced, Extra, year?
S: ------------------------------------------------------------------------
A: ARRL Appointments ? ORS, OES, AEC, VE
A: ARES Equipment ? batteries
E: Emergency Antenna 160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,
E: Emergency Field 160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,
E: Emergency Power 160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,
E: Emergency Power ? 14 days any one freq.
O: Other Emergency ? First aid, CPR, water safety (certs not current)
O: Other Emergency ? German language
C: CW 160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,
F: FM 160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,
R: RTTY 160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,
S: SSB 160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,
M: Mobile 160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,
P: Packet 160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,
B: BBS 160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,
I: Interests-
S: ------------------------------------------------------------------------
C: Call-UP:
S: ------------------------------------------------------------------------
*
S: ========================================================================
S: ******************* New members since January 2001 *******************
S: ========================================================================
* ***end***end*** *
-->
Explaining the Database Structure
More information about a Pfind-database, its structure, and data returns
from searches, can be found in the on-line help supplied with the Pfind-database
application.
The Pfind free-form ASCII database structure only requires the data be
placed in an ASCII file like this:
<!-- CARET DataBase File -->
*
Data Record 1 goes here
*
Data Record 2 goes here
*
Data Record 3 goes here
*
Data Record 4 goes here
*
* ***end***end*** *
-->
You can use the optional command: "endOfList" to "hide" data from the
Pfind database searches. In the example above I used:
<!--endOfList...
and
-->
This hides the database record form. I used for the ARES/RACES data,
both from the browser and from the search functions. The <-- starts an
HTML hide and the --> ends the HTML hide. The "endOfList" hides everything
after it from the Pfind database searching functions, and last, the ...0
was simply used to indicate to me that this was the first time I had tried
to hide this data from Pfind.
To build this type of database you only need a text editor. The "free-form"
part means that you can place the data in the database any way you want.
If you stick to a structure of some pre-determined kind, the Pfind functions
will allow you to do more effective searches; even so, Pfind only allows
basic logical AND searches.
In the example I gave of the ARES database, I placed a number of things
between the database record delimiters (the asterisks: *). I'll try to
explain why I did what I did because there are some special features built
in to the Pfind search functions.
First, if you use a colon ":" as the second character of a line, then
Pfind recognizes this line as a database field line. In a Pfind type database,
each line is a database field and each field is one line long.
In the demo DB I used:
@: for e-mail addresses
P: for phone numbers
C: for the county
E: for some type of emergency equipment or capability, etc.
You can also see that I used E: a number of times so I also put in a
larger “field name” as in:
E: Emergency Antenna 160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221,
445,
E: Emergency Field 160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,
E: Emergency Power 160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,
In a traditional database, if I do a search for all members with the
ability to setup an emergency 40-meter station then I would probably only
get people who had saved “40” in the "emergency field station" database
field. This is not the case in a Pfind free-form ASCII database. One reason
is that I only allow one letter to be used for the field name. In my example
I used " E: Emergency Field" to indicate an Emergency Field operation:
Let's say we have four different members with the following field data:
E: Emergency Field 80, 40,
E: Emergency Field 40, 20, 2
E: Emergency Field 10, 2, 440
and
E: Emergency Field 2, 440 for four different members.
If we now search on "E: field 40", the records for all four of these
members will be returned as hits. (A space between things to search for
means logical AND. The space is NOT preserved as part of the search text.)
One refinement of the search could be using:
"E: field 40,"
Then, only the first two would be returned. In this case you would ALWAYS
have to place a comma after a 40-meter wavelength but not after using
440 MHz to be able to see a difference between "40," and "440".
The reason for this is that the code is designed using the internal JavaScript
methods and I decided not to design methods and functions to implement
a true field-specific database with all its search capabilities.
The rule of thumb for Pfind searches is simply this: if the items being
looked for are on the line (remember, a line is a field), then that line
is a match. Case doesn't matter, position doesn't matter, and order doesn't
matter.
A search can return record data in three ways:
- the matched line (field)
- the entire record
- the address
The street address is a little different in a Pfind database. Pfind treats
all lines as address lines if they do not have a colon as the second character.
This means that when we do a "return address lines only" search, we will
get all lines that do not have a colon in the second character position
as address lines. Also, no delimiters will be placed on the page so you
can cut-and-paste the returned addresses into a word processor to be printed
on labels if you want.
Another feature of a Pfind database is that everything on a line starting
with an asterisk is ignored. Pfind considers that any line with an asterisk
as the first character is a data-record delimiter line, not a data (field)
line. Since it is used only to separate one data record from another,
nothing on this line will be returned as data from any search. So, in
my DB example I used this line to hold an HTML anchor for each amateur
in the DB. HTML does not show anchors on the screen and Pfind doesn't
return delimiter lines so this worked out well. The anchor I used held
the person's call sign, all uppercase.
Another feature is that when a database file is read in, all HTML anchors
are found and an HTML drop-down list is made of the anchor names. In this
example, it was the amateur call-signs.
|
