Tuesday, August 24, 2010

ClearCase Deliver with "unable to compute base" Error

Problem Symptoms:

1. Rebase and deliver operation could not finish on one particular file. Error message:
>>> Operation started: 8/17/2010 3:29:45 PM
Error from VOB database: "\MidTier".
Element "xxxx\InquiriesRequestVOFactory.java", unable to compute base (to \main\Mainline_i\CBFE17.5_THS\djogo_CBFE17.5_THS\29 from \main\Mainline_i\CBFE17.5_THS\36).
Skipping "xxxx\InquiriesRequestVOFactory.java".

Other related errors.
1. The version tree of the file could not be retrieved from GUI with the following error message. But cleartool lsvtree command ran without error.
"failed to retrieve version history of the element"

2. db_server_log error message.
% more db_server_log
2010-08-16T09:17:10-04:00 db_server(15416): Ok:
2010-08-17T12:47:07-04:00 db_server(11335): Error: Database identifier 724419 not found in "../db__obj.c" line 740.
2010-08-17T12:55:13-04:00 db_server(11335): Error: Database identifier 724419 not found in "../db__obj.c" line 740.
2010-08-17T12:57:38-04:00 db_server(11336): Error: Database identifier 724419 not found in "../db__obj.c" line 740.
2010-08-17T13:02:54-04:00 db_server(12862): Error: Database identifier 724419 not found in "../db__obj.c" line 740.
2010-08-17T13:06:08-04:00 db_server(12862): Error: Database identifier 724419 not found in "../db__obj.c" line 740.
2010-08-17T13:06:49-04:00 db_server(12862): Error: Database identifier 724419 not found in "../db__obj.c" line 740.

3. could not merge the file.
>cleartool merge -to InquiriesRequestVOFactory.java -version \main\Mainline_i\CBFE17.5_THS\36
cleartool: Error: Error from VOB database: "\MidTier".

But the file could be merge with only drawing merge arrows.
>cleartool merge -to InquiriesRequestVOFactory.java -ndata -version \main\Mainline_i\CBFE17.5_THS\36
Recorded merge of "InquiriesRequestVOFactory.java".

Workaround.
By drawing an merge arrow as above, we can skip the element for deliver/rebase. but it doesnot resolve the problem.

Diagnose.
1. run command dbcheck.
/etc/utils/dbcheck -r1 -a -k -p32767 vob_db
...
Processing data file: vob_db.d02(3), total of 1302988 records
Problems at record 1298822:
* key field OBJ_DBID(23) error: has a missing key
...
1 error was encountered in 1 record/node

Solution.
1. Stop ClearCase (or untag and unregister the VOB, stop ClearCase and then start ClearCase)
2. Backup (very import!). Copy the db directory
3. Copy the keybuild utility to the VOB db directory
4. run: keybuild vob_db under the VOB db directory
5. Exit out of the db directory after keybuild completes
6. Start ClearCase
7. Reformat the VOB. Run the following command:
cleartool reformatvob

If these steps are completed successfully and without errors, your VOB should now be healthy. If they did not complete or you received errors, restore the VOB from backup.

For my VOB with 7.4GB in size and 5 years in history, it took about 3 hours to finish the procedure.