Bootstrap files & read only mode
Posted 16 May 2000 - 12:31 PM
I have been told (by PROIV LTD) that if you put the bootstrap files into read only mode (i.e. -r--r--r-- in UNIX) then PROIV will not use record locking methods on these files and loading of functions is quicker.
I realise that you cannot then do imports/gens/mods to functions as that was stated too.
Has anyone actually done this, and not seen any problems or have you seen performance gains?
Posted 16 May 2000 - 07:55 PM
I doubt that it's a performance issue - ProIV doesn't do any record locking in lookup mode anyway. I'd be surprised if you saw a performance gain in simple runtime execution but please tell us all if you do!!
I always assumed the reason was so that people could write-protect the bootstraps from unscheduled or malicious changes to their application software. Originally there may have been a problem doing so because ProIV insisted on opening the files in read-write mode, probably now it tries that and then falls back on read-only mode if it fails.
Note that this is just (educated) speculation!
Posted 16 May 2000 - 08:10 PM
What I have been told by PROIV LTD is that....
If you kill iscollect & remove shared memory and then start PROIV any files accessed with -r--r--r-- will not be put into shared memory. Even if the file is open in Lookup mode PROIV still holds an entry in Shared memory. This supposedly speeds things up, I haven't tried it yet though and I guess it would only help on a high volume system.
Posted 17 May 2000 - 09:12 AM
When you read a pro-isam record, a 'currency' is created. It is currencies that you see when you do an isview. It is this same currency that is created for record locking.
I've heard that a currency is not created on a read-only file. Sorry I can't give you 100% confirmation. Let's hope the doctor can confirm.
Even if the currency is not created, this will effect memory and not I/O. A more significant change will occur if you control the I/O.
Some operating systems have a ramdrive facility. The contents of directories/files can be loaded into memory which greatly reduces I/O.
The bootstrap files do represent I/O. An application with a large number of Global functions will have more bootstrap I/O. In screens this is OK but in updates processing high volumes of data, this I/O will impact performance. Reduce the number of globals in these updates to decrease the I/O.
As a minimum, the bootstraps should be stored on a seperate disk to the data. If possible, GENFILE, MSGF and FUNCTDEF should be stored in memory.
The caching options of the O/S will also impact the amount of real I/O as opposed to cache I/O.
Anyone else help ??
Posted 17 May 2000 - 09:29 AM
We currently run about 1 - 1.5 Million functions per day. About 1/4 of these are global functions.
It would be nice if PROIV would cache GENFILE records for global functions, in which case calling a global function would then be as quick as an LSCALL. Currently it is impractical to call a global function 1000's of times in an update.
If global functions were cached in some way (just within the main running function) it would help.
The disk set that our bootstraps are on is in a seperate disk cabinet system (HRZ system) and the main system has GigaBytes of memory so the functions should be in the disk cache, I think.
Guest_Doctor PRO IV_*
Posted 17 May 2000 - 09:32 AM
certain, that no one else can be changing the structure of
the bootstrapfile, then we do not have employ the same level of protection when it comes to simultaneous access to the file by more than one user. We can allow more concurrency between processes contending for the same files such as functdef, vardef and genfile.
The benefits are much more pronounced on a multi-processor
system using spinlocks.
I do appreciate that having read-only bootstraps can cause
operational difficulties but if these can be overcome, then
the benefits may be worthwhile.
Posted 17 May 2000 - 01:39 PM
There is a system analysis tool for most unix systems called 'sarcheck' . A demo can be had at www.sarcheck.com .
I typically notice access times of < 2ms on our cached drives and 14-20 on our non-cached drives.
Sarcheck is great for making suggestions on tuning unix systems.
Posted 17 May 2000 - 06:24 PM
I guess what they're saying is that provided the file is read-only they don't have to worry about reader-processes encountering half-updated blocks and so on and they can dispense with the synchronization which protects the internal consistency of the file. It makes sense that this helps most on a multiprocessor as it'd enable more actual simultaneous processing.
If you're on Unix, you're sure bootstrap access is the source of your performance problem and the relevant bootstrap files are pretty small compared to available memory then you might investigate locking the files in memory using mmap.
I think mmap is supposed to 'map' the disk file to a block of shared memory and provide faster access to it by bypassing the usual Unix filesystem code.
CAUTION: I have never tried to achieve this, I don't know how difficult it is or indeed if it is unsafe for some reason with Pro-ISAM. Also I don't have a Unix system to hand to check if my recollection of mmap is faulty.
Perhaps Dr. PROIV might like to comment.
Guest_Doctor PRO IV_*
Posted 17 May 2000 - 06:41 PM
mmap should be used with extreme caution.
The semantics differ between different versions of Unix and the last time I looked at the Linux version it was definitely broken.
A preferable technique which is just as applicable to VMS is to use a RAM disk.
Linux and VMS definitely support such a feature and other versions of Unix has varying levels of support for such a feature.
Rob's interest in function caching has been noted.
Posted 23 May 2000 - 12:08 PM
I got the following results from testing global functions/read only mode/Mem Disk & spinlocks......
All test cases use 1 LS to read the same record in a for loop 100,000 times. (ie 100,000 acceses of the same record)
All times are averaged over 4 runs.
LSCALL locally in the function
Replace the LSCALL with a GLOBAL_LSCALL
GLOBAL_LSCALL with the 'read only' bootstraps
GLOBAL_LSCALL with the bootstraps on a RAM (memory ) disk with 'writeable' bootstraps
GLOBAL_LSCALL with SPINLOCKS enabled, no ramdisk, writeable bootstraps
I guess that the RAMDISK does not really help as all the records will be cached in memory after the first read anyway.
It looks like the 'read only' bootstraps speeds up intensive function accessing , but I guess caching the GLOBAL LSCALLS would make them as quick as the LSCALL.
Reply to this topic
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users