DYNAMIC FIXED LENGTH STRING ARRAYS

[Dean]

I'm reading .DBF files from disk, and creating an index to the
records in memory. I am storing the fields of the database
file which constitute the unique key to the record in an array
by concatenting the fields into a single string, loading each
string into an array where the number of elements = the number
of records in the file, and then sorting the resulting array. So I
end up with a string array sorted in ascending order based on the
ascii value of each element. I also keep a pointer for each of
these elements to the record number of the file.

Each element of the array is always going to be a fixed length,
since the .DBF file defines fixed widths for each field -- but I won't
know the fields' widths until I open the file and read the .DBF
header, so I can't code a constant width. The value of the width
would never change once I have the file open, but I can't predict
the width of the fields until I open the file. Else, I have to modify
the code for every time file structures change -- not acceptable.

So, depending on the number of records in the .DBF file, and the
number of .DBF files that have to remain open at one time, I need
to conserve memory as much as possible. If my index to the record
is only 15 bytes, I'd like to use up 15 bytes of memory for each
element of the string array.

Dean

[v3cz0r]

Yeah, it could be done with zstring's, and the C's rt lib qsort() could still been used.

Code:

option explicit

#include once "crt.bi"                         '' for qsort() and strcmp()

declare sub fillarray(byval array as zstring ptr, byval strings as integer, byval strslen as integer)

    dim as integer strings = 1024
    dim as integer strslen = 12+1            '' don't forget the null-terminator
    dim as zstring ptr array
    
    '' allocate the array
    array = allocate( strings * strslen )
    
    '' fill it
    fillarray( array, strings, strslen )
    
    '' sort it
    qsort( array, strings, strslen, @strcmp() )
    
    '' print first and last
    print *array; " ="; 0

    print *(array + (strings-1)*strslen); " ="; strings-1
    
    '' deallocate
    deallocate( array )
    
    end 1
    
'':::::
sub fillarray(byval array as zstring ptr, byval strings as integer, byval strslen as integer)
    dim as integer i, j
    dim as string s
    
    j = 0
    for i = strings-1 to 0 step -1
    
        s = str$(i)
        if len(s) < strslen-1 then
            s = string$((strslen-1) - len(s), " ") + s
        end if
        
        *(array + j*strslen) = s
        j += 1
    
    next
    
end sub

I just found out that due a stupid bug at lex.bas, using say ltrim$( *(array + j*strslen) ) won't be parsed, so you have to load the string to a temp string while the unstable/testing versions is not released - i plan to do that this weekend.

[Dean]

Yep - that'll do it. A little more complicated than I am used to,
but basically this is similar to how I am reading the .dbf file itself.
An offset to the starting position of the area I need to read from,
followed by fixed length 'records' (now string elements).

I haven't used memory pointers very much, or zstrings, so I didn't
think of this earlier -- thank you again, and again, and again.

How about the issue with FRE returning different values
after the first time it is called? Not that its much of a bother to
call it twice to get the correct amount of memory (assuming you
really wanted to know the precise amount of memory).
Just thought I'd bring that to your attention, if you weren't aware
of it already.

Dean

[lillo]

Modern OSes use virtual memory: that is, once your physical RAM is all used by apps, and other apps demand for more, the OS doesn't fail (as in the case of old DOS), but instead it used the HD as a new source of memory. Less used chunks of memory are moved to the HD so there's new physical RAM available for the request.
FRE probably returns different results because the OS swaps memory in and out of HD, altering the current physical RAM availability.

[Dean]

Not a big deal -- just being picky I guess, but I don't think
that's it.

I think I just figured it out. The very first time you call FRE
in your program, it's loading the function itself into memory,
or doing something that takes up an increment of 4096 bytes
of memory (its always a multiple of 4096). Run this, and
you'll see the 2nd - 19th iterations return the same number
all the time:

FOR I = 1 TO 20

PRINT FRE

NEXT

INPUT G$


But, do this:

10:

FOR I = 1 TO 20

PRINT FRE

NEXT

INPUT G$

GOTO 10

And now each time FRE returns the exact same value for all 20
iterations after the first time the loop completed.

Who cares -- I'm just having fun. Thanks all for your help!!

Dean

[v3cz0r]

Window's available free memory services are unpredictable, i remember the troubles i had when making the DPMI hack for QB.. every time i called it, more free mem was returned, etc..