Today's Rant #2: How 5 minutes = 1 week
Posted by rekle on Mar 31 2006 in General
Things move VERY slowly in my company. We have one of the most complicated and convoluted release procedures I've ever seen. In order to get even the most trivial 5 minute change into production takes a week minimum. Let's see...
Wednesday, a very minor problem was reported in one the systems I manage. Basically the stored procedure was returning a date string in the wrong format. Ok, so I find the stored procedure, and fix it. It takes all of 5 minutes to find and fix the problem. Here are the steps I have to go through to get this in production:
- Check the file out of Source Safe. Make the minor change and check it in. Make sure to ALWAYS add a comment to the file in Source Safe or any release of this file will get rejected.
- Write a Bill of Materials in Excel listing all the files that changed, what the change is, where the change is going, what the client impact is, who wrote the change, who tested it.
- Fill out a QA database change form, listing in detail every change made to the database. This means list every stored procedure, every table, every field of every table, every key, every index, every database that the change has to be applied to.
- Fill out a Production database change form listing all the same data as the QA database change form.
- Submit the Bill of Materials, and both DB change forms to the Security Audit team for approval. Wait for a response here...
- Once the Security team approves it, they forward the info to the QA/CM department who implements in the QA database. QA/CM also creates a PMF (Production Migration Form) at this point.
- Pass on the changes to your assigned QA resource who goes through and tests the changes in QA. The QA testing approval is then added to the PMF.
- Once it passes QA testing, the changes are then requested to move to the UAT (User Acceptance Testing)
- The QA resource then retests the change in the UAT environment and adds their approval for this step to the PMF.
- Obtain business approval from the project manager and client.
- Submit a requested implementation date/time. Implementations ONLY go out on Tuesday mornings.
- All the changes up to this point have to have been completed by Friday at noon. *any* mistakes along the way and the whole process gets rejected and has to be corrected and started again.
- Assuming all approvals are in by Friday at noon, wait until Monday at noon at which time the PMF either gets approved for implementation or postponed / rejected.
- If all goes well, you show up Tuesday morning at 6am at the data center and wait while they implement your change. Then you and your QA resource test the changes in production. If there are ANY problems, your change is removed and you have to go back and fix it. Which takes another week...
- If you fix a bug in Wednesday, you realisticly have 1-2 days to get it fixed, shoved through this convoluted process, and done by Friday at noon in order to get it implemented the following Tuesday!