Jump to content
Russell Bird

Searching for Data in an external TXT file

Recommended Posts

Hi,

 

I'm hoping to use an external tab delimited TXT to store a master price list for use with our CAD drawings.  The file contains over 12,000 item codes which means I can't load it into a worksheet in VectorWorks as the row limit on those is 4094.  I've proven I can open and read a line from the file, but then I wanted to test how quickly VectorWorks could scan through the rows, so I simply put in a while loop in with an incremental counter variable.  At the file of the while loop (based on EOF), the loop should exit and give me the number of lines read (loops done in effect) by displaying the counter variable in a dialog (using AlrtDialog).

 

I set this running and waited (a long time), but the dialog didn't get displayed and interface was locked out so I assume the script was still running.  2 possiblities: 1) the loop has a fault causing it to loop indefinitely or 2) VectorWorks is really slow at reading lines from a file.  I don't it's option 1 as I've checked the loop code and am pretty sure it's fine.

 

I wondered if anyone could tell me if there are limits to the number of lines or file size that can be read with VectorScript and/or is VectorScript just really slow (in which case pretty useless) at reading files?

 

Thanks

Russell

Share this post


Link to post
Posted (edited)

I strongly recommend switching to python for file access. File reading and data parsing are native to the language, while the VS routines are proprietary and less flexible. (Not to mention, numerous python examples and tutorials exist on the web)

 

https://www.guru99.com/reading-and-writing-files-in-python.html

 

You should be able to read the file and load data into arrays or objects with a couple of lines.

Edited by JBenghiat
  • Like 1

Share this post


Link to post

I agree on the Python idea. And I would suggest using an xml dike instead of a txt file. XML data can be accessed by address instead of by looping.

Share this post


Link to post

if you are in an infinite loop. This could be as simple as checking for NULL data from your read line request to determine EOF. I don't believe that files embed a readable ascii 4 character because EOF is a control state by definition.

Share this post


Link to post

Russell,

   I just finished a script in Python today that imports a 220 MB text file into a LIST (5,321,708 lines) - one text line per list element. I was able to do some serious parsing of the data (extracting names, searching for complex patterns, combining lines, and marking most for deletion), then write it out and back in to repeat the process a dozen times, all in under 35 seconds. This is not possible in VectorScript (VS), as the largest array in VS is limited to 32K elements. You're 12K item codes would fit into a VS array, but you'll have to code most of the routines you'll need yourself. Chances are, what you want to do is already available in Python as a set of canned routines. 

 

   For many things, VS and Python are nearly equal in speed. But if you're going to do a lot of string manipulation, Python has way more text handling features than Pascal. It's worth the effort to explore the language. Also, there is a seemingly unending wealth of help sites and fora on the web for Python. Let Google lead you to the answers you seek.

 

My 2¢,

Raymond

 

 

Share this post


Link to post

Indeed, that is what I would recommend too: use Python. VS was not meant for handling large files.

 

You could even use the csv module, if your data is organized in columns (be it comma separated, separated by tabs or any similar tabular format that the csv module supports).

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


 

7150 Riverwood Drive, Columbia, Maryland 21046, USA   |   Contact Us:   410-290-5114

 

© 2018 Vectorworks, Inc. All Rights Reserved. Vectorworks, Inc. is part of the Nemetschek Group.

×
×
  • Create New...