Back to Rec 3  Help Pages

Problems with Indexes


The indexes in RECORDER can become corrupted if there is a power cut, or if Windows shuts down or crashes with RECORDER still open. You may then get a message similar to the following:

"Link List in the table RECORD LRC RECORDER DATA is broken.
Transactions cannot be properly flushed. Rebuild all the indexes in the table RECORDS."

Whilst rebuilding the indexes might work. (eg go Utilities>Index>Rebuild Index then select the appropriate file) it is likely that they won't rebuild successfully until the broken linkage list is cleared out.

Stuart Ball has offered the following guidance to anyone encountering this problem:

When you ask for an index to be rebuilt, the first thing it tries to do is to process any pending index operations and therefore bumps into the same problem as the background indexer and falls over with the same error! Luckily, it is easy to get rid of the linkage list. In this particular case the error message tells us that it is the RECORDS table which is affected, so we need to delete the linkage list from this table's index, which is in a table called !RECORDS. (pronounced "bang records")

Go to TCL and issue the following commands:

SELECT !RECORDS MATCHING "1N0N"                                (note this is a zero, not an O)
DELETEROW !RECORDS

Now go back to the menu system and rebuild "all indexes" for the RECORDS
table.

This problem usually arises because the machine has been turned off (or
crashed) whilst indexes were in the process of being updated. To avoid such
problems make sure Recorder has been shut down properly before you turn off
your machine.

Stuart



It should always find at least one record with a key of 0 in an index file which will automatically be recreated whenever any R/LIST or indexing operation takes place.

EXPLANATIONS
============

THAT MYSTERIOUS "1N0N"
======================

All Arev users should definitely know about this very useful feature!

The keywork MATCH[ING][ES] in R/LIST allows you to do a pattern patch. The pattern of characters you want to find is described by a series of codes:

nA - n alphabetic characters

nN - n numeric characters

nX - n of any type of character

nZ - up to n of any type of character

So if I wanted to test whether the GRID field contained a 6-figure OS GB grid reference I could use pattern matching as follows:

LIST RECORDS GRID WITH GRID MATCHING "2A6N"

This says find cases were GRID contains 2 alphabetic characters and 6 numeric characters - so "TL207983" matches but "SZ6789" (too few numbers) or "B243765" (only 1 letter at the start, not 2) does not.

This could also be written

WITH GRID MATCHES "2A6N"
WITH GRID MATCH "2A6N"

The "Z" code can be used to find up to a given number of characters, so

LIST RECORDS GRID WITH GRID MATCHING "2A4Z"

Would find 2 letters followed by up to 4 characters and would match 10km (2 numbers), tetrads (2 numbers and a letter) and 1km (4 numbers) grid refs, but would reject 6-figure refs.

The "n" part can be 0 - which means "any number" so:

LIST RECORDS ABUNDANCE WITH ABUNDANCE MATCHING "1N0N"

would find cases where ABUNDANCE consists of one numeric character followed
by any number of numeric characters - i.e. is a straight number and NOT a
DAFOR code, "present", "lots", etc. The reason for using "1N0N", not just
"0N" is that the latter matches any number of numeric characters including
zero. So MATCHES "0N" will match cases where ABUNDANCE is empty. However,
"1N0N" requires at least one numeric char and so excludes rows where the
field is empty.

SELECT !RECORDS MATCHING "1N0N"

You should gather by now that this will find rows with a key which consists
of an integer number. In constructing this command I left out the "WITH
<field>" clause. R/LIST always defaults to working on the key field if
you don't tell it otherwise. I have to do it this way because index files
don't have a dictionary and therefore do not have field names. Consequently
I cannot refer to the key by name and just rely on the default behaviour.

LINKAGE LISTS
=============

When you add or change a record in Arev, the data is immediately written to
the data file, but any updates that are necessary to the indexes are not
made straight away. The system stores the fact that it needs to do some
indexing in the index file. When Arev detects that nothing is happening (no
keyboard or mouse action has occurred for a preset time) then it kicks off
the background indexing process and starts to process pending index
operations - and you see "Indexing" come up in the status line.

When index operations are pending, the necessary information about what
needs to be done is held in records with simple running integer keys: 1, 2,
3 ... in the index file. Operations are written to the last record in the
sequence until it exceeds about 2Kb and then a new record is created in the
sequence.

The system keeps track of them by putting the key of the next record in the
first row of such records. Record "0" points to the first one in the chain.
For example:

Key 0 1 2 3
row 1 1 2 3 0
subsequent rows index operations

The background indexer reads "0" and sees that it must process 1. 1 tells it
to process 2, etc until it gets to 3 which has a 0 in its first row which
means that it is the end of the chain.

This structure is referred to as the "Linkage list".

The background indexer reads 0 which tells it to start with 1. It reads 1
and processes the index operations it describes. It then deletes 1 and
updates 0 so that it points to 2. It keeps doing this until all the pending
operations are done. If this processing is interrupted while the pointers
are being updated you can get in a situation where a linkage records points
to a key that doesn't exist - and this is the dreaded "Broken linkage list"
error.

To fix it, you could:

1). investigate the list yourself by opening each record in turn in the
editor, trying to spot the break and then fix up the preceding record to
mend the chain.

e.g. you find

Key 0 1 3
row 1 1 2 0
subsequent rows ops ops

Record 2 has gone missing and the chain is broken. You could 1 edit it so
that it points to 3 instead of 2 - thus mending the chain.

2). Delete all the linkage list records and rebuild the index as I suggested.

3). Replace the index file with a known good one from another copy of RECORDER (which therefore has an
intact linkage list) and then rebuild the index.

Approach 1 is quick (if you can manage to sort it out!), but you can never
be sure whether the index operations in the missing index record(s) were
properly completed. Therefore you don't know whether the index is properly
up to date - which you can only really fix by doing a full rebuild. The
problem with 2 or 3 is that they potentially take a long time for a big
database.

For example, the MNCR database here at JNCC takes more than a single night
to rebuild the indexes on its biggest table so, when this problem comes up,
I try and fix it using method 1 and then do a rebuild over the following
weekend.

Stuart Ball

 

Alternative method:
Whilst Stuart's method is more elegant and simpler to accomplish than that which follows, the details below are given in case they assist other users in working their way around the insides of RECORDER.

To find which REV*.* files in your RECORDER DATA directory contain the index files run the TCL command:

LISTMEDIA C:\RECORDER\DATA

Look for the Table name !RECORDS and note down the "foreign name" for that Table (eg REV20003)
(See below if LISTMEDIA command doesn't work on your machine)

Using Windows Explorer, go to the C:\RECORDER\DATA and find two files of the same name, one ending .LK, the other, .OV (eg REV20003.LK and REV20003.OV) These are the faulty index files we first need to replace with a non-corrupt version from another copy of RECORDER. and then to rebuild the indexes with.

The simplest source is to find the equivalent files within the Tutorial database. To do this, log on to the RECORDER Tutorial and at TCL type LISTMEDIA C:\RECORDER\TUTORIAL\DATA to find the equivalent files.

Copy these across to the main RECORDER DATA directory, if necessary changing the name to match the original filenames.  RECORDER can now be restarted and the indexes rebuilt.

LISTMEDIA
The TCL command LISTMEDIA is not recognised on my Runtime copy of RECORDER. What do I do?

You will need to create this command before you can use it!
Simply open RECORDER, then press F5
This calls up that blue strip called TCL (usually labeled Command-LRC)
Now type     EDIT VOC LISTMEDIA       I expect it will tell you that
"LISTMEDIA" a new row in the "VOC" Table

Don't worry. If this happens you will need to type in just four lines at the
blank window that appears as follows, starting with the word VERB1 on the
first line:

VERB1

VERBS
V61

having done this, press F9 to save. You can now use the LISTMEDIA command as
described above.
 

Nick Moyes

 

[Home] [Membership] [Committee] [Events] [Newsletter] [LRCs] [Software] [Links]

This site maintained by NFBR website manager.   ©NFBR 2006
Click here for anti-spam and links policy. Scripts are only used in our visitor counter and for minor visual effect on Home and Membership pages.