Jump to content


- - - - -

file size


19 replies to this topic

#1 Guest_Guest_guest_*

Guest_Guest_guest_*
  • Guests

Posted 22 February 2005 - 01:37 AM

Does anyone know the maximum file size for a PRO ISAM file - version 5.5 windows?

#2 Ross Bevin

Ross Bevin

    ProIV Guru

  • Members
  • PipPipPipPipPip
  • 226 posts
  • Gender:Male
  • Location:Manzanillo, Colima, Mexico

Posted 22 February 2005 - 02:46 PM

The maximum ProISAM file size is 384MB. To create a file that can expand to the maximum you need to create it using the following command. This example has a key length of 24.

iscr32.exe -k24 -r3000 test.pro

#3 Joseph Bove

Joseph Bove

    ProIV Guru

  • Members
  • PipPipPipPipPip
  • 756 posts
  • Gender:Male
  • Location:Ramsey, United States

Posted 22 February 2005 - 03:38 PM

Guest,

You may need to check the record length to find the file size.

To do this use the iskeys32 utility.

iskeys32 filename.pro | more

record length: will appear right near the top.

hth,

Joseph

#4 Guest_Guest_*

Guest_Guest_*
  • Guests

Posted 23 February 2005 - 08:29 AM

Guest,

You may need to check the record length to find the file size.

To do this use the iskeys32 utility.

iskeys32 filename.pro | more

record length: will appear right near the top.

hth,

Joseph



When your problem is maximum file sizes you probable don't
want to do iskeys on the file - much simpler to look at the
Pro-IV definition... That goes for 'more' too - look into
'head' and 'tail' unix commands


HTH

#5 Chris Pepper

Chris Pepper

    ProIV Guru

  • Members
  • PipPipPipPipPip
  • 369 posts
  • Gender:Male
  • Location:United Kingdom

Posted 23 February 2005 - 10:03 AM

Personally I use the -s option on iskeys to set a start key past the last key on the file.

So I gues we all have our own methods!!!

#6 Joseph Bove

Joseph Bove

    ProIV Guru

  • Members
  • PipPipPipPipPip
  • 756 posts
  • Gender:Male
  • Location:Ramsey, United States

Posted 23 February 2005 - 03:56 PM

Guest,

When your problem is maximum file sizes you probable don't
want to do iskeys on the file - much simpler to look at the
Pro-IV definition... That goes for 'more' too - look into
'head' and 'tail' unix commands


As far as I know, there is no way to determine the maximum file size of a ProISAM file by looking at the file definition. The reason is that when the file is created, any record length may be specified. It is the record length at file creation time that determines the maximum size of the file...

The record length recorded within ProIV is just the total number of bytes per record.

Regards,

Joseph

#7 Guest_Guest_guest_*

Guest_Guest_guest_*
  • Guests

Posted 23 February 2005 - 05:24 PM

Guest,

When your problem is maximum file sizes you probable don't
want to do iskeys on the file


As far as I know, there is no way to determine the maximum file size of a ProISAM file by looking at the file definition. The reason is that when the file is created, any record length may be specified. It is the record length at file creation time that determines the maximum size of the file...

The record length recorded within ProIV is just the total number of bytes per record.

Regards,

Joseph

Hi ,

The recordlength value returned by examining the file is normally the same as that in the file metrics in P4.
Unless of course it was created manually using iscr or sysf...

But I do have an issue with this:

The record length should reflect the required length for the file...

file size = (keylen + recordlen) * number of records

max file size = 384 MB = (MAXkeylen + MAXrecordlen) * number of records

n = 384000000 / (127 + 3000)
n = 122,801

well that's crap

to add more records to the file reduce the recordlen or the keylen

setting -r3000 should merely allow a record to be that long - the actual records may not be nearly so long.

i..e you should not get more records into the file by just increasing the max record length.

Of course I think I'm wrong as I know people commonly let files grow by recreating them
with -r3000 option - so what's going on when that is issued? Why does
increasing the recordlength allow more records?

#8 Guest_ProISAM, Ha Ha HA HAA_*

Guest_ProISAM, Ha Ha HA HAA_*
  • Guests

Posted 23 February 2005 - 07:00 PM

:x: For ProISAM, I thought ProIV artificially adjusted the "record length" to a power of 2 but perhaps that's my misunderstanding.

The -r option doesn't really set a record size at all, it determines the BLOCK size of the ProISAM file and hence determines the max file size. I think the blocksize is simply taken to be twice the notional recordsize.

As with many dumb things in ProIV, the total number of blocks in a ProISAM file is probably limited to a 16-bit quantity (ie. 64K total blocks). Hence 64K (blocks) x 3Kb (so-called recordsize) x 2 (records per block) = 384Mb

ProISAM Records are always variable-length anyway so if you have big blocks you can obviously fit more records in your file.

#9 Joseph Bove

Joseph Bove

    ProIV Guru

  • Members
  • PipPipPipPipPip
  • 756 posts
  • Gender:Male
  • Location:Ramsey, United States

Posted 23 February 2005 - 07:23 PM

Guest,

The -r option doesn't really set a record size at all, it determines the BLOCK size of the ProISAM file and hence determines the max file size.


This is my understanding too. We have a special tool for our Unix clients that relies on the information from iskeys to determine what the actual maximum file size is. This is then compared to the actual file size and a percentage of use calculated.

Then, to make things very easy for the client, a report is generated if any files are over 80% of capacity. This has saved us a number of crashes and emergencies since we implemented it.

Unfortunately, we've not built the same for Windows or I'd gladly share it.

Regards,

Joseph

#10 Chris Pepper

Chris Pepper

    ProIV Guru

  • Members
  • PipPipPipPipPip
  • 369 posts
  • Gender:Male
  • Location:United Kingdom

Posted 24 February 2005 - 08:41 AM

OK, a quick history lesson on why Pro-Isam exists at all.
When ProIV was first put on the PC it used C-Isam. In the begining under PC-DOS a program was restricted to using just a few file-handles (16 or 32 or something like that). Since C-Isam needs 2 per file this meant that ProIV pretty much ran out of files using the files needed for Run-time, let alone development. So a programmer quickly put together a simple B-tree file system that stored both the index and data in a single file thus doubling the number of files that could be accessed.

Since the amount of storage was effectively limited by the hardware available at the time, accessing huge amounts of data wasn't a requirement.

ProIsam was just one of those accidents of history.

The basic design limits the number of blocks, so increasing the block size is the only way to increase the overall file size.

I agree with Joseph that 80% is a good "warning level". In the past I've noticed that, probably due to the way the B-tree grows that a file can suddenly grow at a much faster rate as more records are added. Clearly it depends on the spread of the keys.

#11 Dan Shannon

Dan Shannon

    ProIV Guru

  • Members
  • PipPipPipPipPip
  • 374 posts
  • Gender:Male
  • Location:Australia

Posted 24 February 2005 - 01:30 PM

Just an unreasonable opinion based on nothing but snobbery:

The maximum size for a PRO-ISAM file in any self-respecting production application ought to be 0 bytes.

I'm in a bad mood, sorry.

#12 Donald Miller

Donald Miller

    ProIV Guru

  • Members
  • PipPipPipPipPip
  • 205 posts
  • Gender:Male
  • Location:Cupar, Fife, Scotland
  • Interests:Motorcycling, Running, Cooking

Posted 24 February 2005 - 01:53 PM

So is it still true that Pro IV is more efficient, if the record size is large, that you keep the whole record within one block ?

ie. if the record size is 600 bytes and the block size 500 then re-creating the file with a larger block size will keep things efficent.

Does the block size have to be a factor of 8/16/64 ?

Cheers
Half of what he said meant something else, and the other half didn't mean anytthing at all

#13 Joseph Bove

Joseph Bove

    ProIV Guru

  • Members
  • PipPipPipPipPip
  • 756 posts
  • Gender:Male
  • Location:Ramsey, United States

Posted 24 February 2005 - 02:14 PM

Dan,

The maximum size for a PRO-ISAM file in any self-respecting production application ought to be 0 bytes.


That creates quite a burden for logic.pro and genfile.pro! :x:

Regards,

Joseph

#14 Mike Schoen

Mike Schoen

    Expert

  • Members
  • PipPipPipPip
  • 198 posts
  • Gender:Male
  • Location:Guelph, Canada

Posted 24 February 2005 - 03:03 PM

I got these numbers from pro-iv years ago, when we had some proisam files explode.

Page Length Maximum Size
----------- ------------
512 32MB
1024 64MB
1536 96MB
2048 128MB
2560 160MB
3072 192MB
6144 384MB


We have written a global function to always create files with a page size of 6144 (equivalent to iscr -kN-e 3000),
and we run scripts unix systems to parse ischk -v output and warn of file size issues based on these numbers.

Doesnt help if a customer ignores the report, but enough people catch it to make it worthwhile.

#15 Chris Pepper

Chris Pepper

    ProIV Guru

  • Members
  • PipPipPipPipPip
  • 369 posts
  • Gender:Male
  • Location:United Kingdom

Posted 24 February 2005 - 05:19 PM

So is it still true that Pro IV is more efficient, if the record size is large, that you keep the whole record within one block ?


No, a requirement of ProIsam is that you must always be able to get at least 2 records in a block. AFAIK you will get an error message if you try to write a record larger than a block. ProIsam cannot cope with 'block spanned records'.



Reply to this topic



  


0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users