8 replies to this topic

[Rick Young]

seeking opinion please - or even better, fact

Which "costs" more (processor and/or time-wise): calling a logical set of nested cycles/LU's, or calling a GU to perform the nested cycles/LU's? Approx .5mill calls will be made. Only "good" reason I currently have for wanting to use a GU, is so I don't accidentally screw myself with key/field values reading the same physical & logical filedef as in the main cycle/LU.

Thanks in advance
Rick

[Lewis Mccabe]

Rick,

5 million calls - I cannot imagine doing it any other way than cycles within function (no global). Use alt file to avoid variable collision. But you already probably knew that.

There is one thing you might investigate - we don't use it but function caching may help. Someone else may have additional information on that one.

Lew

[Rick Young]

Hi Lewis

Thanks for your reply. It's only 0.5 million calls each making a totally unknown number of reads. This is envisaged to be a "one time" deal, so maybe "cost" doesn't matter quite so much, upon reflection vs the human cost (mine) of creating a not insignificant number of extra alt-filedefs.

Have you - or has anyone - done any "clinical math" on this general scenario? I'm thinking maybe I should one day just to get a better feel for it.

I have read a little on that function caching in the doccos - but am going to wait until my client gets a secondary server (lesser spec than primary) up and running before I think about experimenting with it.

Cheers
Rick

[Lewis Mccabe]

If this is a one time and time won't be critical, just go with a global function. Regarding alternate files, what version are you on that alt files cannot be automatically created? If it is really old, be prepared for time consuming global function calls. From version 4 to 6 i know this is an area in which performance has been improved a lot. We tried to do a global call on a paging screen in version 4 without success. Performance dropped to a crawl. That same function structure on version 6 is quite acceptable.

Lew

[Rick Young]

Regarding alternate files, what version are you on that alt files cannot be automatically created?

6.2. I guess it depends on what "automatically created" actually means...for this one-time deal, I have created a few alt filedefs for some of the files, and a quick'n'dirty utility to rename the field names in filedef and vipbm25. I'm basically - perhaps somewhat oxymoronically - doing "extra work" in the name of safety but being a lazy b*stard don't want to do too much extra work...

Under some time pressure (i.e. by about next Tuesday) to get this done (including testing) - but at the same time, going for a little education I'm pretty much done now - onto some testing soon.

Here's the guts of the scenario:
- client is pure pro-isam
- have 1 progdir and multiple datadirs, each with a different @COMP
- based on some user-defined simple and complex rules, have to pull some data from each of the datadirs, and drop them into yet another datadir with yet another @COMP
- I have a personal dislike of screwing with @COMP (call it paranoia)
- data is always going from FILE-A in datadir1,2,3 to FILE-A in new datadir
- hundreds of fields involved - some being directly imported, some "if rules" based on source datadir, some arbitrary hardcodes, and some "auto numbering"

I do have to say, I doubt there would be any way possible for me to get this done in the timeframe involved, if I weren't working in ProIV - perhaps even if I were working with an RDBMS. $0.01


Thanks for yor thoughts
Rick

[Lewis Mccabe]

I agreee with you about speed of PROIV in this instance. That being said, I don't love doing these kind of projects. Looks as if you are well on your way. Good luck.

Lew

[kapoof]

Hi Rick,

As reagrds to auto automatically creating file defs, use the "child file defs" in VIP. After this change FILEHDR @ALFNAME to the required path name (ProIV runtime uses the original bootstraps, at least most of the when running). This will allow you to quickly move your data and also create the file defs. Hope this helps.

Best Regards,
Rob.

[Rob Donovan]

Hi,

Yep, as others say, use an alternate file def and a local LU.

Global Functions have a massive overhead on performance, and unless you can justify that by making it a GF and it reduces your dev alot (is in, its used in multiple places), then its just not worth it.

Dont get me wrong, GF's have their place, but its a fine line that you have to look at with GFs, with performance vs development ease.

The problem with them is the way they've been implemented. Not only do you get the hit with having to load the function (functdef, genfile, securty, vardef, msgf), but also you lose all the performance savings that a local LU gives you. For instance, in a local LU the file is only opened and setup once, even though its called lots of times, but if you move that to a GF, it opens, stats, reads the header and closes the file every call, so it can hurt alot with performance.

I've looked briefly at GF caching, but to be honest its not 'that' good. Firstly, its only genfile, so the bottle neck moves to another of the runtime files, and secondly its something you have to pay more for

Another thing that can help, with GFs, it to make you BootStrap files Read Only, then P4 skips alot of the locking and shared memory stuff with them, and this can have good effects on performance, but it needs some thinking about a bit, as you cant then make any changes while the system is 'live'.

I've done alot of looking into ProIV performance, and I'll try to get a post up soon, that shows all the pretty figures and graphs and stuff regarding this, and some recommendations.

Rob.

[Rick Young]

Thanks for the input guys. I must confess, I "knew" the answer, but after RD's additional info, I didn't know it was "that big" of a hit. However, for this one-time import deal, I still chose to use the GU route for my own peace of mind to not lose the plot with any @COMP or fieldname issues. Customer reviewed the rules, and elected to cut down on a date range thereby excluding several million reads (i.e. 0.5 mill x "lots") - runtime is about 10-minutes. I may rewrite it all as an LU one day, for comparison purposes.

@kapoof/Rob - I haven't as yet played with Child filedefs (though was aware of their existence), and given time constraints, didn't feel it was a good time to learn about them - they do seem useful though.

@RD - I have considered readonly bootstraps - and in the perfect world of never Production coding or hot-fixing it'd be great - but...can't say never at this client's site

Cheers
Rick