<< Comments on DNR #226: Entity Framework | Home | Unit Testing Trivalities? >>

That might take a while...

Yesterday I started a run of a process that should run every 5 minutes, I got annoyed when it took too long and went home. I just got back to see this number:

SSIS_ORA_ENT.png

Maybe I need to rethink the architecture. It is still working on it. The really scary thing is that the select count(*) that I also left running didn't finish in ~16 hours or so. And just to clarify, this is an OLTP DB, not a data warehousing one.d

Print | posted on Wednesday, April 11, 2007 7:44 AM

Feedback

Gravatar

# re: That might take a while... 4/11/2007 8:11 AM SSIS Guy

On large tables, it's usually more efficient to put the data flow in a for loop and chunk through it in blocks of (50k, 100k, etc.) rows. If you've got a sequential PK, then this is pretty easy.

It's much easier for the query processor to deal with smaller chunks of data at one time, and SSIS likes it better too.

Gravatar

# re: That might take a while... 4/11/2007 8:32 AM Ayende Rahien

I didn't have a clue that the table was this big. I will probably need to rethink the way we do it.

Thanks for the tip

Comments have been closed on this topic.