EngageOne Vault loader process explained

Vault for Windows versions 5.4, 5.5, 6.0, 6.1, 7.0
When a Vault load is successful (no errors) this is the type of logging you'll see:


14:45:40 open log, date [2014/07/31], file [D:\Group1\e2\Vault\server\log\e2loaderd.20140731.144540.7220.log]
15:17:30 moved resource pack [work\jobextractossolarisvault.hip] to [resource\jobextractossolarisvault.hip]
15:17:30 moved resource pack [work\mt8.him] to [resource\mt8.him]
15:17:35 have journal [work\portafoliovtcv20140725.jrn], waiting for compressed file
15:17:35 compressing [work\portafoliovtcv20140725.afp] profile [PORTAFOLIO] format [AFP]
         0    10   20   30   40   50   60   70   80   90   100
         |    |    |    |    |    |    |    |    |    |    |
         XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
15:20:05 pages: [168233] total read: [1070065518]  total written: [117375357]  final ratio: [9.12:1]
15:20:05 file [work\portafoliovtcv20140725.afp] has been compressed
15:20:05 build [work\portafoliovtcv20140725]
15:20:05 profile [PORTAFOLIO] method [uxmljournal]
         0    10   20   30   40   50   60   70   80   90   100
         |    |    |    |    |    |    |    |    |    |    |
         XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
15:20:32 processing resource packs for guids [439137CA89CF44B3B8A48B993B51348A,1E3AD64D7EBE4AA6A3F05F28F570B621]
15:21:08 [76988] documents, [168232] document pages, [0] ignored pages
15:21:08 renaming file [portafoliovtcv20140725] to [20140727002720-portafoliovtcv20140725-afp-ae11ec02154e11e488fe8db884ec858a]
15:21:08 start indexing document file [docdata\20140727002720-portafoliovtcv20140725-afp-ae11ec02154e11e488fe8db884ec858a.drd]
         0    10   20   30   40   50   60   70   80   90   100
         |    |    |    |    |    |    |    |    |    |    |
         XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
18:20:12 flushing index cache
18:20:12 index\portafolio\account.dri
18:20:12     file size [10451718144]
18:20:12     disk read [23]
18:20:12     disk write [907]
18:20:12     stack depth [6]
18:20:12     cache read hit [775806]
18:20:12     cache read miss [22]
18:20:12     cache write hit [77875]
18:20:12     cache write miss [887]
18:20:12     add unique [76988]
18:20:12 index\portafolio\identi.dri
18:20:12     file size [5935992832]
18:20:12     disk read [141428]
18:20:12     disk write [77176]
18:20:12     stack depth [5]
18:20:12     cache read hit [213821]
18:20:12     cache read miss [141427]
18:20:12     cache write hit [77497]
18:20:12     cache write miss [509]
18:20:12     add unique [76988]
18:20:12 index\portafolio\invlink.dri
18:20:12     file size [39739670528]
18:20:12     disk read [2597]
18:20:12     disk write [5937]
18:20:12     stack depth [8]
18:20:12     cache read hit [562661]
18:20:12     cache read miss [2596]
18:20:12     cache write hit [80352]
18:20:12     cache write miss [3364]
18:20:12     add unique [76988]
18:20:13 finished indexing document file [docdata\20140727002720-portafoliovtcv20140725-afp-ae11ec02154e11e488fe8db884ec858a.drd], [0] errors

Here's a details explanation of everything that is happening with this load.

When the e2loaderd service is started (either via the Windows Services control panel or via the command line, this line gets written to the log: 

14:45:40 open log, date [2014/07/31], file [D:\Group1\e2\Vault\server\log\e2loaderd.20140731.144540.7220.log]

When files created with later versions of Doc1 are ingested the HIP file (also known as the resource pack in earlier versions) gets processed and moved to the vault/server/resource directory:

15:17:30 moved resource pack [work\jobextractossolarisvault.hip] to [resource\jobextractossolarisvault.hip]

If Content Author is being used with the job, the HIM file will also be processed, and moved to the vault/server/resource directory:

15:17:30 moved resource pack [work\mt8.him] to [resource\mt8.him]

Vault checks to make sure that it has the journal, and looks for a print file that has the same metadata and file name as the journal:

15:17:35 have journal [work\portafoliovtcv20140725.jrn], waiting for compressed file

When it finds the print file (in this case, AFP, it compresses it.  In this case the total time to compress the file was two minutes, 30 seconds (from 15:17:35 to 15:20:05)

15:17:35 compressing [work\portafoliovtcv20140725.afp] profile [PORTAFOLIO] format [AFP]
         0    10   20   30   40   50   60   70   80   90   100
         |    |    |    |    |    |    |    |    |    |    |
         XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
15:20:05 pages: [168233] total read: [1070065518]  total written: [117375357]  final ratio: [9.12:1]
15:20:05 file [work\portafoliovtcv20140725.afp] has been compressed

In the section below, Vault builds the DRD file.

15:20:05 build [work\portafoliovtcv20140725]
15:20:05 profile [PORTAFOLIO] method [uxmljournal]
         0    10   20   30   40   50   60   70   80   90   100
         |    |    |    |    |    |    |    |    |    |    |
         XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

 Here Vault cross checks the print stream, journal, and HIP/HIM to make sure that they all match (are from the same print run)  Unless explicitly overrridden in profiles.ini settings, the GUID must match between the print file, XML journal, HIP file and HIM file.  Note that there are two guids below, one for the HIP and the other for the HIM.  If only a HIP were used, only one GUID would appear below:

15:20:32 processing resource packs for guids [439137CA89CF44B3B8A48B993B51348A,1E3AD64D7EBE4AA6A3F05F28F570B621]
15:21:08 [76988] documents, [168232] document pages, [0] ignored pages
15:21:08 renaming file [portafoliovtcv20140725] to [20140727002720-portafoliovtcv20140725-afp-ae11ec02154e11e488fe8db884ec858a]

Vault then indexes the document.  Note that in this case, it takes around three hours (15:21:08 to 18:20:13)

15:21:08 start indexing document file [docdata\20140727002720-portafoliovtcv20140725-afp-ae11ec02154e11e488fe8db884ec858a.drd]
         0    10   20   30   40   50   60   70   80   90   100
         |    |    |    |    |    |    |    |    |    |    |
         XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
18:20:12 flushing index cache
18:20:12 index\portafolio\account.dri
18:20:12     file size [10451718144]
18:20:12     disk read [23]
18:20:12     disk write [907]
18:20:12     stack depth [6]
18:20:12     cache read hit [775806]
18:20:12     cache read miss [22]
18:20:12     cache write hit [77875]
18:20:12     cache write miss [887]
18:20:12     add unique [76988]
18:20:12 index\portafolio\identi.dri
18:20:12     file size [5935992832]
18:20:12     disk read [141428]
18:20:12     disk write [77176]
18:20:12     stack depth [5]
18:20:12     cache read hit [213821]
18:20:12     cache read miss [141427]
18:20:12     cache write hit [77497]
18:20:12     cache write miss [509]
18:20:12     add unique [76988]
18:20:12 index\portafolio\invlink.dri
18:20:12     file size [39739670528]
18:20:12     disk read [2597]
18:20:12     disk write [5937]
18:20:12     stack depth [8]
18:20:12     cache read hit [562661]
18:20:12     cache read miss [2596]
18:20:12     cache write hit [80352]
18:20:12     cache write miss [3364]
18:20:12     add unique [76988]
18:20:13 finished indexing document file [docdata\20140727002720-portafoliovtcv20140725-afp-ae11ec02154e11e488fe8db884ec858a.drd], [0] errors

That's it!  The document has been loaded successfully.
UPDATED:  December 24, 2019