Importer 2.x

Legacy documentation for Importer 2.x.

Importing and processing data can take up a fair amount of resources on your server. When your data source is a file, the larger the file and the more complex its structure, the more resources will be used up. Because of this, there might be a limit to how large of a CSV, JSON, or XML file your server might be able to import using the Importer module. The resource most likely to be used up is the amount of memory allocated to PHP. We have a couple tips to increase this limit.

Dynamic Memory Increasing

First, Importer has a built in method that will dynamically attempt to increase the memory allocated to PHP during the import, if it detects that the import is about to use up the available memory. The maximum limit that Importer will attempt to raise the allocated memory to is 500MB, which is to insure that importing does not cause problems with your server. The increase is temporary and will only last the length of the import. This feature will only work on servers that allow it and will happen seamlessly behind the scenes. It will not guarantee that large imports will perform flawlessly, but it should help.

Increasing Memory in .htaccess

Some hosts will allow you to increase the PHP memory limit in your .htaccess file. Check with your host first and insure this is allowed. Also, see if there is a maximum limit they recommend. Other people may be hosted on your server and it would be inappropriate to continually use up large chunks of memory.

If this is allowed, it is a rather simple. Simply add this to the top of your .htaccess file:

php_value memory_limit 256M 

You can see your current memory limit by looking in the Tools => Utilities => PHP Info section of the ExpressionEngine CP. Do a quick search for memory_limit and you should locate it. NOTE: The M on the end stands for MB. Insure you use M though as PHP will not recognize MB.

Increasing Memory in php.ini

If you are hosting your own server or are in control of your own server, you likely can modify the php.ini file, which controls all of PHP's default configuration options. You can find its location by looking at the Tools => Utilities => PHP Info section of the ExpressionEngine CP. Again, talk with your server administrator before modifying any values in this file.

To modify the memory limit, do a search for memory_limit and modify the value. If there is a hash (#) before the line, you will have to remove it. Once you make the change and save the file, you will have to restart Apache to have the change take effect.