Forum

I must import a large txt document with 256 columns and 200,000 rows. I use the TextReader method, but it last very  long. Can you help me with other idea of making the import so it work faster?

If executioning context is Windows, then you can use MS FileSystemObject,

for example:
* bugt: removed by administrator. non-english sites links

Total line:

    fso = NewCOMObject ("Scripting.FileSystemObject");
    file = fso.GetFile(stringFileName);
    
    TextStream = file.OpenAsTextStream(1, -1);
    //file = fso.OpenTextFile(stringFileName, 1, , 0);
 
    While file.AtEndOfStream=0 Do
        EachLineOfTextStream= TextStream.ReadAll();
        LetsHandlThisLine(EachLineOfTextStream);
    EndDo; 
 
    TextStream.Close();

I imported txt documents with 600,000 row and 55-60Mb size. I didn't have problem to read txt document. I used "TextDocument" method. I had another problem with the recording speed of the data into the database.
Do you write those data into the base ?

In fact, the problem is saving/updating in the database, not reading the text file.
It is any posibility to bulk insert / update a catalog in the database with data from a text file?

Actually it depends on many factors - first of all - what kind of data we want to ins ert in database. If it's InformationRegister - you can read whole text file, and insert it in database at once, in one RecordSet.
For AccumulationRegister may be more subtle to invoke Se tTotalsUsing(false) before writing data in database.

The file is more a catalog than an InformationRegister. It is in csv format.
Because the file is very large and we get out of memory, we actually process it reading groups of lines, check if the records are already in catalog an update or insert if it is not there. But this takes very long time.
Any sugestion is verry wellcome.

#6, what exactly takes long time - can you elaborate the whole process with more details?

Did you try the TextReader object?

It was designed for sequential read text files with long length.

Text = New TextReader("d:\win.txt");