How can we help you today? How can we help you today?
Charles
Thanks, Richard. I increased the timeout to an hour, and it works fine on smaller tables. Now... I have a very large table (over a billion rows), of which I'm trying to compare a relatively small subset (2.3 million rows). I have a non-clustered index built on two fields (sourcetype & sourceid). I added a WHERE clause to the compare to define a range on this index, and I'm only asking it to compare these two fields. can do a count(*) on the table using this where clause, and it uses the index and does the deed in under 3 minutes. (If I include another field not in the index, it plans a table scan and forecasts a ridiculous cost) The checksum method times out after an hour (because that's what I set the timeout to), showing no progress. I never noticed any progress on smaller tables, either, but they finished (e.g. just did a 17-million row table in 3 or 4 minutes) Is it normal for no progress to be shown during the checksum stored procedure run? I've seen this, where with a "normal" compare progress is displayed normally. Of course, now, I killed the SQL process (because cancel didn't do the deed in a reasonable amt of time) and restarted a "normal" compare with the same parameters, and it's showing 100% done while it works... but anyway, it's running the SELECT that I expect to see, and cancels okay (I tried it again to see if it did the same thing) so maybe the progress display just doesn't like this table... actually, now that I think more about it, the progress display is probably based on an estimated rowcount for the table from the statistics, so a WHERE clause makes it impossible to use that. What kind of performance improvement have you seen on larger tables using the checksum vs. non- method, anyway? (assuming relatively few differences; otherwise I'd guess it would actually be longer) Any Thoughts? Is the reason for the timeout on the checksum method that the checksum procedure does nothing for a long time and then returns an answer list where the "classic" SELECT trickles data out as it goes? I would think that an hour would be enough for either method, but maybe I need to increase the timeout more and be patient... :-( / comments
Thanks, Richard. I increased the timeout to an hour, and it works fine on smaller tables. Now... I have a very large table (over a billion rows), of which I'm trying to compare a relatively small ...
0 votes
I'm sorry, Richard, I totally missed your post right before Kevin's! My fault. richardjm wrote: I was confused by your comment of 'Needless to say, this freaked me out big time until I figured out that it wasn't real.' - what do you mean it wasn't real? Machine A is replicating to machine B. I knew that there were records on A that never made it over to B (92k, as confirmed later by doing the Key-only compare), but it was rather shocking to see it initially report not only 500k of those, but another 600k that supposedly existed on B but weren't on A! (I also spot-checked those records and verified that they DID indeed exist on A, contrary to what it claimed.) richardjm wrote: Also can I check you're using SQL Data Compare 6. Yes, build #1124. BTW, I like the photo that appears after 30 seconds in the about box. It's a bit classier than one of my products where it brings up a cheap pong game with everyone's heads as the balls. 8) richardjm wrote: If you can send me your .sdc project files Okay. I glanced through it and didn't see anything conclusive. I'll pass it on to you. richardjm wrote: You can run a profiler on the database in both situations That's a good suggestion. I also might play with some other things to simplify it and see where I can get the problem to drop out. For example, I'm using a compound key, but my WHERE clause limits the 1st field of the key to a single value, so there's no real need to use that field in the join. Anyway, I'll need to wait until tonight, because the queries murder my data load (in addition to guaranteeing inconsistencies). Thanks again! / comments
I'm sorry, Richard, I totally missed your post right before Kevin's! My fault. richardjm wrote: I was confused by your comment of 'Needless to say, this freaked me out big time until I figured ou...
0 votes
Charles wrote: BTW, It's a bit classier than one of my products where it brings up a cheap pong game Oh, how fun -- it *is* a game! Turns into a little puzzle after a while. I like it! / comments
Charles wrote: BTW, It's a bit classier than one of my products where it brings up a cheap pong game Oh, how fun -- it *is* a game! Turns into a little puzzle after a while. I like it!
0 votes
I'm experiencing the same thing, though this is the 1st situation where I know for sure it's happening. I'm comparing two tables with identical schemas, one on the local (2005) machine, and one on a remote (2000) machine. I'm using a 2-field index as a key, and because the tables involved are large (1.2 billion rows) and the suspect portions are relatively small, I'm using a WHERE clause on the key to limit the area checked. (Yes, this is the same pair of tables I discuss 3 threads after this one) When I limit the fields to compare to JUST the keys, the comparison results are thus (a only, diff, b only, same): 92599, 0, 0, 566192 (total 658,791) This is what I expect to see. However, if I allow it to match ALL fields, then these are the numbers I get: 636788, 36, 544189, 21967 (total 1,202,980) Needless to say, this freaked me out big time until I figured out that it wasn't real. So, at this point, I can't use the program to sync these two tables; all I can do is use it to generate inserts for the keys, send it to a file, and manipulate it to build a list of keys to feed into a query to bcp out of the one database and into the other. The behavior is duplicable every time right now; I'd be happy to work with you in the next couple of days to track down what's causing the glitch. BTW, after reading another post I think I should add --though it shouldn't have anything at all to do with it -- that the 2005 table is partitioned, the 2000 is not. All data being compared is in the same partition. / comments
I'm experiencing the same thing, though this is the 1st situation where I know for sure it's happening. I'm comparing two tables with identical schemas, one on the local (2005) machine, and one on ...
0 votes