How Long Does a Purge Project Take?

March 31, 2015

Authored by Alana Smith, TeamCain

It is the age old question that has plagued IT crowds for many decades. Well, maybe not decades per se... And I guess the item that has plagued IT crowds is when they have to ask “Did you try turning it on and off again?” Nonetheless, this is the subject that can add some worry to an already swamped IT employee. The fear is that a purge project is a neverending project and that the initial purge of the years of data will take forever. This really isn’t the case.

The question of how long a purge project will take is really dependent on the company that is purging the data and how much data there is to purge. One manufacturer’s purge story was that they archived 130GB of data (that was 61% of their JDE data) in 25 days. Their data conversion time took less than 36 hours. An American shipping company was able to archive 6 years of data in a weekend and removed 300GB of data across four environments. Both these examples show that no matter the industry, we all need to purge and each purge project duration is going to depend largely on the amount of data the company has to purge.

We asked our Purge-it! customers in a survey how long their initial purge and archive project took and 58% reported that it took them less than a month (less than two 2 weeks was another popular reply). How might this be possible? It’s not like one just drops everything else and performs a data purge. That’s why a lot of customers like Purge-it! because you can purge while users are on the system. The purge and archive routines run just like any other batch job that you submit to the enterprise server. Therefore, you can run them at the same time that the users are on the system.


"During the purge and archive process, Purge-it! identified that we had a lot of data that was not closed, highlighting problem areas in our data. As well as the core purging and archiving processes, we also went through a data tidying process to archive as much data as possible, so the overall process took a few months end-to-end, but if we had not cleansed the data at the same time then the main purging and archiving process would have been much quicker." - International IT Project Manager, Global Communications Company

The initial purge project itself will happen while you’re working on other tasks – it just runs in the background. What will take up your time is if you encounter any bumps. Bumps include data that hasn’t been closed, configuration settings - such as security (as an aside, Purge-it! installs as an ASU), and deciding on just what you should be purging.

The biggest bump that you’ll encounter that can delay a project is getting decision makers to decide on how much data to keep in production, to keep in the archive, and to just get rid of. Users can be quite protective of data and so it’s important to show your users that they needn’t worry. Rarely have we seen situations where there is real, defined and justified need for years and years of data. From the user’s perspective, there never will be a good time to purge and archive the data they need. Be prepared to ask “why” - most of the time, there are alternative ways to handle what users deem as required data. 

So once you’ve dealt with user delays and set up the environment, your purge project will be complete before you know it!

You Might Also Be Interested In:
Indi-Alana Jones & the Raiders of the Lost Archived Data
Spring Cleaning: Data Purging Best Practices (Part One)
Are you about to embark on a data purge project? Here are 6 key things to factor in