5. Ongoing management for exporting to WorldCat
There are some ongoing requirements to manage the Automated Export to WorldCat process.
During uploads
WorldCat issues folders
During each upload session, a Titles folder is created which contains title records for which there was a problem uploading to WorldCat data sync. It is called WorldCat Issues: YYYY-MM-DD where YYYY-MM-DD is the date on which the folder was created. This should be reviewed regularly to pick up any problem titles and modify them accordingly so that they do not fail next time the upload is run.
A “template” folder is supplied, called "WorldCat Issues: "YYYY-MM-DD, which the system uses to create the new "real" folders. Permissions can be set on this template folder, e.g. to limit access to the folder to the system administrators. These permissions will be copied automatically to the new "real" folders when they are created.
Configure summary email alert
The export to WorldCat process can be configured to send an email alert with summary results of the processing, when the exchanges with WorldCat have completed. Normally you would have completed these when configuring the Output Format record.
Once the upload has run, the email is generated.
Example email
Adjusted file numbering to start at: 1
Listed 75687 titles
Exported 3 into File No 1
File /metacoll/in/bib/1013406.OURLIB.bibholdings20181117.01.mrc is .00MB
Completed 3 titles (0 failed)
1 Books/Monographs (Survey report)
1 CEH Staff publications
1 Cartographic materials (Maps)
75683 were not exported
73319 were not suitably authorised
2364 had no public Copies
75683 had no Objects (or Digital/Film Notes) Checking for Batchload response files Reading Cross-Ref file: metacoll.OURLIB.datasync.1013406-1013406.D20181108.T230822450.xrefrpt.txt
Reading Cross-Ref file: metacoll.OURLIB.datasync.1013406-1013406.D20181113.T225412874.xrefrpt.txt
Reading UDEV file: metacoll.OURLIB.datasync.1013406.D20181112.T171006000.bibdetailexcpt.1.mrc.txt
Processing UDEV Files
Processing Cross-Ref Files
During the last execution 3 titles were processed.
Of these, 3 were synchronised OK and 0 did not.
If the MARC record generation fails, the problem titles will be placed into a folder for that day, for example, Title Export Issues: 2017-01-08. As with the WorldCat Issues folder, a Title Export Issues “template” folder is supplied, called "Title Export Issues: "YYYY-MM-DD. The system uses this template folder to create the new "real" folders. Permissions can be set on this template folder, e.g. to limit access to the folder to the system administrators. These permissions will be copied automatically to the new "real" folders when they are created.
Status report
On the Output Format: Email Generation sheet you can see a status report for exported records under the Data Push Specific settings heading:
| Field | Description |
|---|---|
| Alert if no Records Processed? | Set this flag to Yes to generate an email if no records have been exported. |
| Alert if no Records Failed | This will generate an email if all records are successfully exported. |
| Current OPAC Status Summary | This provides a summary of the current status for all titles in the title set and any that have been removed from the title set. Each status is a hyperlink which will display a list of the title records at this status. |
| 7-Day Status Summary | This provides a summary of the current status for all titles that have changed their status in the last 7 days. The hyperlinks to the title search do not limit the search to the last 7 days. |
For information on how the status changes during the typical lifetime of a title, refer to the OLIB to WorldCat Title Lifecycle.
Title records audit trail
The Titles: Audit Trail sheet is populated with changes to the export status, e.g.:
| Change By | Changed Field | Old Value | New Value |
|---|---|---|---|
| System Administrator | ISxN | 1283375192 | 9781283375191 |
NOTE: The update process is fully automatic. If a catalogue record is changed, OLIB will automatically include the changed record in the Update file when it is next scheduled to upload updates.
The Audit Trail sheet on the catalogue record will show changes to the export status. For example, RTP (Titles with this status are due to have an update sent to the external system during the next daystart run.)
Processing WorldCat data sync responses
Response files
Once WorldCat data sync has processed a file of bibliographic records, up to 3 Response files will be generated and processed automatically:
Cross-Reference file
(Example file name: metacoll.OCWMS.datasync.1003600-100360.D20160323.T124708500.xrefrpt.txt)
This file contains:
- The OCLC Numbers for the matched and new records
- The title number from OLIB
OLIB will import this cross-reference file, during which it will assign the OCLC Record Numbers to the titles in OLIB and mark the records Updated. The bibdetailexcpt file includes Severe (logical), Critical (structural) and Minor responses. Titles that have either Severe or Critical responses will be marked Rejected.
Titles may be marked as Rejected for the following additional reasons:
- More than one OCLC Number has been supplied for a single title*
- The same OCLC Number has been supplied for multiple titles
- The OCLC Number supplied is already assigned to another title
- The title has been archived
UDEV file
This is the Bibliographic Exception Detail file. (Example file name: metacoll.OCWMS.datasync.1002106.D20160729.T082400.bibdetailexcpt.1.mrc.txt)
- This file contains information on issues, possibly errors, with the data supplied
- If the record listed in here has anything other than a Minor issue reported, it will be flagged as Rejected
- Details are imported into the title_discovery_system_report table
Unresolved file
(Example file name: metacoll.OCWMS.datasync.1003600-1003600.D20160323.T124708500.unresxrefrpt.txt)
- This file contains record numbers (both OCLC number and the title number from OLIB) that were not processed due to significant errors
- If the record listed in here is not included in the Cross-Reference file above, it will be flagged as Rejected
When a record is rejected by this processing, it will be placed into a “WorldCat Issues …” folder for that day and will have an internal note assigned providing further information.
If OLIB cannot determine a response from WorldCat data sync for a particular record, it will remain at Sent to WorldCat, Holdings Pending until Retry Days, whereupon it will be sent again.
More details are available in My Files reports.
Manual Import of Cross-Reference Batch Record
If you need to import the WorldCat data sync cross-reference file manually (e.g. because it did not get picked up by the automated process owing to network connectivity issues), you will need to set up a WorldCat data sync CrossRef import batch record.
- Go to Data Exchange>Import Batch and click New Record.
- Edit the following fields:
| Field | Description |
|---|---|
| Batch Type | WorldCat data sync CrossRef. This will cause OLIB Web to change the layout. |
| Description | Enter a description |
| Output Format | Select the WorldCat Update output format record that you created for synchronising your data |
| Files To Import | Attach the cross-reference file in the normal way, using the Octet stream object type and uploading it into the database |
- When you have attached the File To Import, check the box to select it, and choose the Process action from the drop-down.
- From the Alert window choose Load to actually process the file
If OLIB cannot recognise the file pattern of the cross-reference file, it will leave the following message on the object record attached to the Import Batch record:
The Batchload response file type was not known. Only ".CXREF", ".UDEV" or ".UNRES" are currently handled
Frequency of updates
After the successful initial upload, you should check the frequency of ongoing updates to WorldCat. The upload of new and updated records to WorldCat data sync is done daily in Daystart. (Normally you will have already set this when configuring the Output Format record).
- Go to Data Exchange Reference Data> Output Formats.
- In the Export Generation sheet, ensure that Frequency is set for the Holdings maintenance to synchronise with WorldCat for deletions and further additions. OCLC recommends a 30 minute setting for updates once everything is in place. (Do not use the days, weeks, months or years options for this purpose.) Alternatively, if you want an email for every run, set it to around 6 hours.
Deletions
- Record deletions are notified to WorldCat automatically when the record is deleted or archived, via the OLIB-WorldCat Metadata API Link.
- If the last copy of a title is deleted but the title itself is not deleted, a Holdings Removed deletion notification is uploaded to WorldCat automatically, again via the OLIB-WorldCat Metadata API Link.
Information in the Title record
OPAC Status fields
In Related Items and Control Data, the OPAC Status fields are also relevant:
| Export Target | Status | Last Change by | Last Change Date |
|---|---|---|---|
| WorldCat Update | Remove Record / All Holdings | System Administrator | 28-Nov 16:19:29 |
> OPAC Status Codes, Descriptions and Notes
Available Actions
When you check the box to select your export record, a number of Actions for OPAC Status information become available to you in the drop-down:
| Action | Description |
|---|---|
| (re)Populate Push Title List | This should be for system administrators only. It is used to reset the push status for all titles in order to send everything again. |
| Change Push Status | This enables the status for be changed to: Remove Record / All Holdings, Ready to Update or Refresh Holdings(?) for this one title for this specific OPAC target. |
| Synchronise with OPAC now | This is pertinent for the WorldCat metadata API and CONTENTdm and is used to send the update or delete message to the target OPAC. |
| Verify Status Locally |
This will examine the status of the record and set it to a suitable value to facilitate sending an update to the target OPAC. This is useful for e.g. changing Rejected (because the username and password was wrong, but has now been corrected) to Ready to Update. |
Discovery system report
A list attribute in the Titles domain allows you to view the response details. The attribute is called Discovery System Report. The list attribute needs to be added to the Titles layout using Layout Manager. (Suggested placement: Control Details sheet, close to the OPAC Status information).
This shows each line of the report from data sync of the current Title record you are viewing.
Example data
| Date | OCLC Number | Severity | Message |
|---|---|---|---|
| 04-Sep-2017 | 991639940 | Severe Logical Error | Invalid subfield 1st $u in 1st 650 |
| 04-Sep-2017 | 991639940 | Severe Logical Error | Invalid subfield 2nd $u in 1st 650 |
| 04-Sep-2017 | 991639940 | Severe Logical Error | Invalid subfield 3rd $u in 1st 650 |
This list attribute shows the date and time of the processing of the response, together with the OCLC Number, the severity and message information.
To view and process the WorldCat responses:
- Go to Titles and retrieve the relevant record.
- In the sheet containing your OPAC information, review the responses in Discovery System Report.
- Make changes to the Title record if required, depending on the severity / message.
- When you tick the box to select a response, you can use the Delete link for that line. This allows you to remove each line once you have corrected it in the catalogue record
Refresh holdings for a specific OCLC symbol
It is possible to refresh all holdings for a particular OCLC symbol from OLIB. To use this refresh requires logging in to sqlplus as the configuration user. Please contact the Support Desk if you want to carry out this process.