ccroft Posted July 3, 2005 Share Posted July 3, 2005 Is anyone else having trouble with this? Just updated to 11.5 from 8.5 and my worksheets aren't summarizing correctly anymore. I have a number of them that list rectangles by class, and their height and width. In a drawing with say 3 rectangles of the same size, the worksheet should say 3 fronts 24x36. Instead I get: 1 front 24x36 1 front 24x36 1 front 24x36 It'll do this even if the 3 are all sized by OIP. I just did a test where the 3 are duplicates and the list was correct...until I moved them around. That's in a new drawing (not a template) with a worksheet built from scratch. This creates a lot of extra work and more opportunity for costly error. Is it a bug? Thanks for any help. Charles [ 07-02-2005, 10:06 PM: Message edited by: ccroft ] Quote Link to comment
ccroft Posted July 3, 2005 Author Share Posted July 3, 2005 There was some discussion on the v-scrpt list about internal rounding/tolerances. Is it related? Ray Mullin posted "BewareCOMPARISONoperators" on Feb 9 2005. Quote Link to comment
MullinRJ Posted July 3, 2005 Share Posted July 3, 2005 Hi Charles, What happens if you export the VW11.5 file back to VW8.5? I'm not a guru of worksheets, but it may help if you list your summary formula. The error I reported earlier had to do with numbers being compared to 0. If they are any closer than about ?10^-13, comparison operators declare them to be equal (i.e., (3*10^-15 = 0) evaluates to TRUE). Sometimes this is a good thing, sometimes not. It depends on what you are doing and how picky you need to be. I don't think your problem is related, but I can't tell yet. Raymond Quote Link to comment
ccroft Posted July 3, 2005 Author Share Posted July 3, 2005 So....I wrote a little script to message out the results of HHeight of these little rectangles. The message says they're all 30. If I export vectorscript, the object creation code for the rectangles shows to 15 decimal places. Subtracting Y2 from Y1 is never exactly 30. Moving them around changes the result of Y1 minus Y2 in the last decimal. A totally inconsequential distance to me, unless this affects the comparison operations used in the sort routines of the database summary. Could it be that the summary I'm trying to use is evaluating on the co-ordinate system rather than the displayed values? (just wildy guessing) Quote Link to comment
ccroft Posted July 4, 2005 Author Share Posted July 4, 2005 HI Raymond Good to see you. I exported back to V8.5. Worksheet displays un-sorted until re-calculated, and then displays correctly.The formula is very simple: List Objects whose Class is front =COUNT =HEIGHT (summary widget) =WIDTH (summary widget) Other worksheets which summarize some objects by their name do so correctly. I'm kinda grasping at straws here and wondering if it has something to do with the floating point change and higher internal accuracy that happened aroung V9. I'm assuming that =HEIGHT returns a real (like HHeight). The summary/sort will compare the height and width values and increment the count if they are equal. If, for some reason unknown to me, the values are not exactly equal to the final decimal point, then the count won't be changed. The displayed numbers are equal. Changing the number format has no effect. If an object is sized by OIP and this value is held internally as a Real to 20 decimal places (or whatever our current accuracy is), will the final place always be zero? Thanks for your interest Charles Quote Link to comment
MullinRJ Posted July 5, 2005 Share Posted July 5, 2005 Hi Charles, I just tried what you did and get similar results for some Rects, and expected results for others. When exported to v8, I also get good and bad results, though the counts are different than in v11 (go figure). It does look like a rounding problem, but it's been here a while. Try these formulae: =INT(ROUND(Height*1000))/1000 =INT(ROUND(Width*1000))/1000 to give you 3 decimals of accuracy. Change the number of zeroes in the scale factor (1000 in this example) to get more or less decimal precision. Raymond [ 07-05-2005, 11:46 AM: Message edited by: MullinRJ ] Quote Link to comment
ccroft Posted July 5, 2005 Author Share Posted July 5, 2005 Thank-you sir! A bit more testing and I'll be comfortable with it. So if I'm working to 1/16th inch and displaying dimension numbers in the worksheet, it looks like 10 is the right scale factor. Does that make sense to you? Thanks again for taking the time.This is a very big deal for me and I was starting to think I might have to go back to 8. Maybe I should take some math classes. Charles Quote Link to comment
MullinRJ Posted July 6, 2005 Share Posted July 6, 2005 Hi Charles, You are welcome. If you are working in binary fractions, then use powers of 2 as you would powers of 10 for decimal precision. Or, more to the point, make your scale factor = 16. The ROUND function will snap things to the nearest 1/16. Actually, the INT function in my previous post is redundant. ROUND is similar to INT but has a little more savvy. =ROUND(HEIGHT*16)/16 =ROUND(WIDTH*16)/16 If you put your scale factor in a spreadsheet cell and reference it, you can change it easily. For example, set cell A1 to 16 then use: =ROUND(HEIGHT*A1)/A1 =ROUND(WIDTH*A1)/A1 Then change it to 8 to see if your counts change. They should if there is enough variation in your data. Try 4, 2, 1 & 32 for comparisons, or amusement. HTH, Raymond Quote Link to comment
ccroft Posted July 6, 2005 Author Share Posted July 6, 2005 Beautiful! As I said, I need some math tutoring.....and I seem to be getting it! This is the stuff the real programmers all know. I'm just a self taught vector-scripter. I was preparing to make 3 or 4 databases,sorts and worksheets with V-script. Not an insignificant piece of work for me even tho it's been done before. This is SO much easier, AND updates automatically. Indeed, as the scale factor goes higher, the summary gets messier and messier. So I think it's nailed. Thanks again, and I hope you keep hanging around here. Charles Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.