Ayende @ Rahien

Unnatural acts on source code

That might take a while...

Yesterday I started a run of a process that should run every 5 minutes, I got annoyed when it took too long and went home. I just got back to see this number:

SSIS_ORA_ENT.png

Maybe I need to rethink the architecture. It is still working on it. The really scary thing is that the select count(*) that I also left running didn't finish in ~16 hours or so. And just to clarify, this is an OLTP DB, not a data warehousing one.d

Comments

SSIS Guy
04/11/2007 05:11 AM by
SSIS Guy

On large tables, it's usually more efficient to put the data flow in a for loop and chunk through it in blocks of (50k, 100k, etc.) rows. If you've got a sequential PK, then this is pretty easy.

It's much easier for the query processor to deal with smaller chunks of data at one time, and SSIS likes it better too.

Ayende Rahien
04/11/2007 05:32 AM by
Ayende Rahien

I didn't have a clue that the table was this big. I will probably need to rethink the way we do it.

Thanks for the tip

Comments have been closed on this topic.