It is not easy to answer this question. There are many things should be done to get performance gain.
Yes, to get more speed you should go away from data caching, increase amount of memory that could be used by PHP
like ini_set("memory_limit","256M"); or directly in php.ini
Also, you can try to use different PHP accelerators like APC, eAccelerator, etc.
But the most important thing you should do is to measure your code execution time part by part. Probably you will see that save time does not take so much time than smth else.
In my project I did not get any performance gain using "fromArray" but in your case it might help.
You can also check your data to avoid creating of empty cells.
I think it is not an option but I see that my project is working faster on PHP5.2 than on PHP5.3.10. (have not tried on PHP5.4 yet)