Mass User Profile Import problems with Drupal

Post by Mike C

Reading time 1 mins

Drupal 5.x and 6.x both suffer from static variable problems in the Forms API which causes huge problems with memory usage whilst processing large amounts of data during a single-pass import.

During a recent project we needed to import 25,000 user profiles in to a Drupal powered site, creating a login and populating a profile node for them.

notes:-
A lot of drupal_execute() calls are used by modules such as usernode and node_profile.
static variables in the validation code causes problems.
chunking csv data stops memory usage.
clear drupal messages regularly.