Are you about to embark on a data purge project? Here are 6 key things to factor in...

April 23, 2012

Authored by Sarah Mack, TeamCain

Have you tried a purge before, or considered a purge, but were held back by concerns about integrity or completeness? You are not alone – our surveys show that this is one of the main reasons why organizations don’t purge their JDE data.

Our experts recommend purging and archiving every couple of years at a minimum, and for sure if you have at least 7+ years of data. Heading into an upgrade?  You really do want to look at archiving and purging… Your upgrade “to the 9s” will go a whole lot quicker.  But the thought of a purge project can be daunting. Here are six key questions to ask yourself when about to embark on a data purge project to make sure that your project goes smoothly.

What “date horizon” should be used?

Before starting your purge project, you must consider your timeframe - usually this is based on the data retention policy that is set up in your organization. You have to factor in legislative, business, and legal requirements as well as consider a historical comparison (i.e. what information have you needed in the past that is important for the future?). As a general rule of thumb, you want about 2 years plus the current year in your production environment – the rest can go to archives. In your archive, you want about 5 years of data. All in all, a total of 7 years on your system is sufficient to making sure you are adhering to any requirements and keeping your system up to date. When thinking about a purge project, make sure to approach your application users as they know what they need from the data. Again, this is a “rule of thumb” – it is not uncommon from what we have seen for customers to only keep 5 years of data in production and archive. 

Which modules should you purge, and how will you decide?

Approaching a purge project is like cleaning out your closet of your old clothes. You love that sweater from 5 years ago, despite the moth holes. You keep telling yourself that those neon pink leg warmers will come back into style. Not all modules are equal and lots of applications run in JD Edwards. How do you decide what to get rid of and what to keep? Begin with looking at the most heavy transaction files (AP, GL, Work Orders etc.) - which of those modules are critical to your business? The next step is to analyze how much space each application uses and which one uses the most. Take a look at your custom files as well – how do they work with JD Edwards, how are they linked with JD Edwards, will you need to purge these files along with the JDE files? This part of the purge project requires analysis, some investigation and a little work before jumping in. You want to make sure you didn’t accidentally get rid of something important. Help from experts in analyzing your data files and determining where to get the most “bang for the buck” often is a good idea.

Do you have custom tables and/or add on applications?

Having add-on applications and custom tables is very common among JDE customers – these are usually the F55 – F59 series files and applications. Many have custom development for specialized purposes like extended SOP or specialized CRM. You need to keep these in mind when approaching your purge project.

Who will need to be involved?

When thinking about who will be involved in the purge project, you need to first assess the big picture. Your business people will be busy and your system itself will be busy running jobs. You need to budget time for internal staff to be away from their daily job, other projects that may be impacted, end users review the purge results and time for any data quality issues review. Normally, the technical staff and user staff are involved in a purge project. Those who work with the applications need to be involved as well so they can say yes this is okay to purge and make sure that has been purged and archived is correct. Remember, the business users are the real owners of the data!  When we do purge projects with our customers, there are 3 main levels of staff and external involvement: 

  • We want to do it all ourselves as much as possible.  This is a great approach if you have the staff and time … you are up to speed and have the internal knowledge then to carry on subsequent purge and archive projects.
  • We want to be involved but don’t have the time to do most of it ourselves.  This is likely the most common scenario we encounter.  You set up the project so that you get the knowledge you need to be self sufficient but don’t cramp your staff time.
  • We want to get it done, but don’t have the time ourselves.  Busy days, busy people … this is not uncommon.  Bringing in a partner who understands JD Edwards, your needs, and purging can make this a simple project.  BUT – your business users still need to review and sign off on what will be purged (and archived).

How will you maintain the data integrity inherent in JDE?

To say that accountants and users “kind of” care about data integrity is a bit of an understatement! JD Edwards is all about tight integration, flexibility and user control. You want to be careful that if you are taking data out of one application that it is looking at all the files relevant to that application and not leaving anything behind. If it did leave items behind, your data integrity could be compromised. You want to make sure that the users are verifying the integrity of the data that is being purged, so make sure to run JDE integrity before and after the purge project.

What will you do with the archived data?

Now that you have successfully completed your purge project, what do you do with all that data? There is a lot to consider when answering this question. First, you need to think about if a team is going to go back and look at the archived data or if you want to take the data right off your system right away. You also need to think about if perhaps keeping some of the data on your system will be sufficient enough in case you need to access it. You need to factor future purges into your place – do you keep the archived data until that next purge project? As there are many involved with purge projects, you need to consider how often you will purge, who will be on that project team, what is going to be purged and how far back you are going to go. You can always purge the archived data at a later date if you wished.  However you do it, having simple access to archived data makes your users a whole lot happier. 

Find out what almost a decade of purge and archive experience with over 100 customers can mean for your project – contact our experts today.

We do have an elegant solution to your purge and archive problems – find out more here.