Vault Error 70129 -Problem with the index - When searching for a file

On this case:

Vault : 5.5M0p0312 / Windows 2003

Issue

If they search (assuming some problem with the index) 

09:50:43 127.0.0.1:4216 <connection1> database.search request, prefix [8165285694], database [dbof_fixa_g003], index [csac], max [20]
09:50:43 127.0.0.1:4216 <connection1> database.search returned record, [6] rows, [12] columns, more [0], fixed [9], elapsed [0], cached [1670]
09:50:43 127.0.0.1:4216 <server1> remote host closing connection
09:50:43 127.0.0.1:4216 <server1> close connection, requests [3], read [327], written [3411], duration [3] ms
09:50:43 [1] listening, [0] open, [0] stopping, [0] stopped, resetlevel [0]
09:50:45 127.0.0.1:4217 <server1> open connection
09:50:45 127.0.0.1:4217 <authserver1> database.list request
09:50:45 127.0.0.1:4217 <connection1> database.list request
09:50:45 127.0.0.1:4217 <connection1> database.list returned record, [28] rows, [2] columns, elapsed [0], cached [1689]
09:50:45 127.0.0.1:4217 <authserver1> database.list returned record, [28] rows, [2] columns, elapsed [0], cached [1689]
09:50:45 127.0.0.1:4217 <connection1> database.info request, database [dbof_fixa_g003]
09:50:45 127.0.0.1:4217 <connection1> database.info returned record, [7] rows, [5] columns, elapsed [0], cached [1682]
09:50:45 127.0.0.1:4217 <connection1> database.search request, prefix [000004551410969], database [dbof_fixa_g003], index [account], max [30]
09:50:45 127.0.0.1:4217 <connection1> database.search returned record, [2] rows, [6] columns, more [0], fixed [3], elapsed [0], cached [1650]
09:50:45 127.0.0.1:4217 <connection1> database.info request, database [dbof_fixa_g003]
09:50:45 127.0.0.1:4217 <connection1> database.info returned record, [7] rows, [5] columns, elapsed [0], cached [1682]
09:50:45 127.0.0.1:4217 <connection1> database.info request, database [dbof_fixa_g003]
09:50:45 127.0.0.1:4217 <connection1> database.info returned record, [7] rows, [5] columns, elapsed [0], cached [1682]
09:50:45 127.0.0.1:4217 <connection1> database.search request, prefix [], database [dbof_fixa_g003], index [invlink], max [999], under [000004551410969]
09:50:45 <connection1> connected to [10.61.176.168:6001]
09:50:45 127.0.0.1:4217 <connection1> <storage1> ERROR 70129: unable to read file attributes for file [docdata\20140430-pr-ftd05-d201405-g003val-ofic-s001-062234-20140430.drd], error [2]
09:50:45 127.0.0.1:4217 <connection1> database.search failed, status [70129]
09:50:45 127.0.0.1:4217 <server1> remote host closing connection

Cause

The search 

database.search request, prefix [], database [dbof_fixa_g003], index [invlink], max [999], under [000004551410969]

will try and return all records associated with account 4551410969. If it is not a lightweight search (and it will not be for this vintage) then each matching index record will cause the associated drd file to be opened to get things like page count, etc. This is the essence of the FAT lock problem. If one of those files is missing, then the I suspect the entire search will fail. 

09:50:45 127.0.0.1:4217 <connection1> database.info request, database [dbof_fixa_g003] 
09:50:45 127.0.0.1:4217 <connection1> database.info returned record, [7] rows, [5] columns, elapsed [0], cached [1682] 
09:50:45 127.0.0.1:4217 <connection1> database.info request, database [dbof_fixa_g003] 
09:50:45 127.0.0.1:4217 <connection1> database.info returned record, [7] rows, [5] columns, elapsed [0], cached [1682] 
09:50:45 127.0.0.1:4217 <connection1> database.search request, prefix [], database [dbof_fixa_g003], index [invlink], max [999], under [000004551410969] 
09:50:45 <connection1> connected to [10.61.176.168:6001] 
09:50:45 127.0.0.1:4217 <connection1> <storage1> ERROR 70129: unable to read file attributes for file [docdata\20140430-pr-ftd05-d201405-g003val-ofic-s001-062234-20140430.drd], error [2] 
09:50:45 127.0.0.1:4217 <connection1> database.search failed, status [70129] 
09:50:45 127.0.0.1:4217 <server1> remote host closing connection

Vault cannot find the file 
20140430-pr-ftd05-d201405-g003val-ofic-s001-062234-20140430.drd

Resolution

UPDATED: September 25, 2017




-Check on the docdata if the file  is not there, in this case file: 20140430-pr-ftd05-d201405-g003val-ofic-s001-062234-20140430.drd

-If the drd file is truly no longer there, then they will have to clean up their index to make it match the drd files that  are there.

-Verify if there is a problem with the server disk corruption first, this issue needs to be corrected before the re-index.
Check with developers which process best fix for the index.


In this case developer advised on the index:

I would normally say -validfile would be enough but as they are aggressively pruning, I am concerned that the index searches will run into that performance issue of accessing a lot of empty nodes. They can certainly benchmark it with a small number of accounts to measure how long it takes."

"
 Given that their I/O subsystem does not look good, they may want to wait on that."