Export to a third party discovery service
A number of libraries that use OLIB also use a discovery service provided by a 3rd party supplier, for example EDS from EBSCO. This information outlines how to configure OLIB so that it automatically exports records to the 3rd party discovery system, including both the initial “full” export and regular “update” exports, which will include details of modified and deleted records.
Create a title set
A Title Set should be created to determine which records are to be exported to your discovery service. A separate "export" title set is created to allow for differences from the titles made available in OLIB, WebView or Folio.
Go To Data Exchange> Title Sets and click New Record.
Fields to note:
| Field | Description |
|---|---|
| Search Level/Type | Set to OAI Service to ensure that it does not conflict with the WebView/Folio title sets |
| User Set |
Set to Default All Users and Anonymous Users Set |
| Include / Exclude | Use with Copy Locations / Copy Categories / Copy Statuses as required, so that you export the correct sub-set of records |
Save the details. Next, go to the hitlist and search to obtain your new title set. Check the box in the hitlist to select it and from Other Actions choose Refresh This Title Set.
OLIB responds:
The delimited OPAC view for this title set has been refreshed - OK
Configure MARC tag for ebooks
Some third party discovery services (e.g. EBSCO) require your MARC records to be exported with e.g. Electronic Books in the 655 tag.
Electronic Books will be included with the MARC export (tag 655) when it runs as part of daystart, or when you run the export manually.
Create subject "Electronic Book"
A Subject Type of Form is available in OLIB. Create a Subject Heading of e.g. Electronic Books, giving it the Subject Type of Form:
(The key ID is _FORM and it is only created if your system has no existing Subject Types with 655 in the Tag field).
- Go to Cataloguing> Subjects Headings.
- Using the standard Subjects layout create a new Subject record Electronic Books.
- Select the Subject Type of Form from the Type drop down.
- Save the details.
- Go to Cataloguing Reference Data> Media Types (or Media Subtypes) and search for your ebooks record.
- In the MARC Details section, search and select the Default Subject of e.g. Electronic Book or eBook as created above.
- Save the changes.
When you view an E-book catalogue record in the Titles domain, you will see that the MARC Details sheet includes the tag 655=Electronic Book:
655.#/1 $aElectronic Book
OLIB will include this when the export is run as part of Daystart or when run manually.
Create an Output Format
In the Output Format Details sheet:
| Field | Description |
|---|---|
| Description | Enter a name for this output format – e.g. EDS Export/ftp |
| Domain | Set to Titles |
| Type | Select Export To External System by FTP |
| Location |
Use this to determine which information from the copy records that are linked to the title record is exported. If a location is added to this field, that location’s copy records will be exported in full to tag 852, and other location’s copies will generate a 952 tag with whatever is defined in the Copies.MARC21 Configurable Copy Output record if it is enabled. If the Location field is left blank, all locations’ copies will be exported in full to tag 852. |
| Call Ref. |
This is automatically populated – Do Not Edit this field |
| Format Text | Enter an additional description here, e.g. EBSCO EDS Export/ftp |
Details in the Export Generation sheet determine the items to be included in the export and the frequency of updates:
| Field | Description |
|---|---|
| Title Set |
Select the title set created for export to your discovery service |
| Exclude Record Source |
Use this to exclude records imported into your OLIB catalogue which have been supplied by the discovery service, so that records are not duplicated. For example ebooks supplied by EBSCO if you are using the EDS Discovery service |
| Accession Date Between / And |
The export process will only export records where the accession date has been populated and falls within the date parameters specified. In order to include records in perpetuity, do not enter an “end date” |
| Accession in the Past nn Days | Not currently implemented / not required |
| Folder | |
| Frequency |
If you are configuring the output format record for an initial full export or to re-export the full data set, set the frequency to 1 weeks. This gives you added flexibility for checking the output. If you set Frequency to 1 days and you don’t get a chance to check the output on the day that it is exported, the following day’s export may overwrite this full export. Once a full export has been completed and checked, change Frequency as required to specify how often a file of updates is to be exported. This can be specified in days, weeks, months or years. We would recommend a daily setting for updates. The export process runs during daystart, so the minutes and hours frequencies should not be used. |
| Enabled | Set this to Yes to export records |
| Start on |
Select a date when you would like the first export to be generated |
| Last Output On |
This is populated automatically with the date of the last export. When generating a full export set the Last Output On date to the earliest Accession Date. |
| Output Next On |
This is populated automatically according to the parameters set in the Frequency. You can manually edit it as required. Note: when you initially set up the output format for a full export, or if you are re-doing a full export, enter the date that the first export is to run. If no date is entered initially, no records will export. It will be subsequently updated according to the frequency parameters. |
Next complete the FTP details required for export to a 3rd party discovery service:
| Field | Description |
|---|---|
| Files Sent and Retrieved |
This is a read-only field. It will contain the link to the file that is sent to the 3rd party discovery service. |
| Limit 952 Locations | This option will limit the copy information included in the data supplied to the FTP server to copies for those locations that are included in the title set. This requires usage of a MARC21 Configurable Copies Output Format. |
| Hierarchical Copies Handling |
This option allows control over which copy information is supplied for multi-volume records. The option most suitable for your chosen discovery system should be used. This requires usage of a MARC21 Configurable Copies Output Format |
| Track Status? |
Set to No to ensure a full supply of all titles every time data is sent. (OLIB will not mark the titles as sent). This option will prevent other libraries who share your OLIB system from being hindered if they attempt to delete titles that have mutual copies. |
| Retain Sent Files? | Set to Yes to keep a copy of the latest file of records sent to the 3rd party discovery service. This is useful whilst setting the ftp service up in the first place. Once the service is up and running, this field can be set to No. |
| Uncompressed Local File Size Limit | Set both fields to 2000000 |
| Uncompressed Target File Size Limit | |
| Withdrawn Folder |
If a Title is modified such that it is no longer included in the associated title set (for example, all the copies at the included locations have been deleted) and Track Status? is not set to No, then the record will be sent as a Deletion to this target system. This will be done without marking the title itself as a Deleted Title as it may be still required by other locations. When this occurs, the title will be placed into the folder that is specified in this field. This is the only way to identify such titles without advanced searching and review of the title audit data. The link to the output format is deliberately removed so that, should a copy be re-acquired by one of the included locations, the title will be re-sent automatically to the third-party system as a new record. |
| FTP URL |
Enter details of the ftp site which will be supplied by the discovery service supplier. Make sure you precede the ftp address with ftp://. For example, when configuring the output format record for a full export: ftp://ftp.tdnet.com/full/<YYYY>-<MM>-<DD> and when configuring it for an update: ftp://ftp.tdnet.com/update/<YYYY>-<MM>-<DD> Note: the FTP process will not create directories on the FTP server. These must already be present. This means that you cannot send data to a directory with the date in the name unless something else has generated the directories first. Note also that you must not use “/” as the date element separator, as this is the directory separator character, i.e. <YYYY>-<MM>-<DD>… is OK, but <YYYY>/<MM>/<DD> is not. |
| FTP Username and Password | Enter the username and password as supplied by the discovery service supplier |
Once a full export has been supplied the FTP URL field should be amended to supply files to the update directory as shown here:
ftp://ftp.epnet.com/update/<YYYY>-<MM>-<DD>
Configure a confirmation email on the Email Generation Details sheet, that will be sent on completion of the export.
- From Address, Email To, must be entered for an email to be sent.
- Enter CC List and BCC List email addresses as required.
- Enter a Subject Line for the email.
(Email Message is used in an alternative export procedure and can be ignored)
(re)Populate Push Title List
This is an Action you take in the Output Formats hitlist and should be for system administrators only. It is used to reset the push status for all titles in order to send everything again. It should be used after saving the Output Format (with the Title Set linked) and before Daystart runs to send the data:
- Save the Output Format record (with linked Title Set) as above.
- Display the Output Format record in the hitlist.
- Check the box to select it and choose (re)Populate Push Title List.
On occasion this may be required for re-sending the full catalogue.
Note: do not schedule the process to send to a file in the “full” folder every day, as it will only send changes and the receiver might then remove most of the data from your discovery system. Once a month (or longer) is recommended so that you can change it back to doing updates at your earliest convenience - rather than risking something preventing that change the following day (e.g. system failure) and the “full” folder then getting compromised.
Managing the export
Once the export has run, the email set up is generated. An example of the email generated would be:
Exported 51 into File No 1
File EDSTestMARCFile04-1.txt is .00MB
Completed 51 titles (0 failed)
Go to the Email Generation Sheet of the Output Format. A status report for exported records can be found under Data Push Specific settings:
| Field | Description |
|---|---|
|
Alert if no Records Processed? |
Set this flag to Yes to generate an email if no records have been exported |
| Alert if no Records Failed? |
If set to Yes, this will generate an email if all records are successfully exported |
| Current OPAC Status Summary |
This provides a summary of the current status for all titles in the title set and any that have been removed from the title set. Each status is a hyperlink which will display a list of the title records at this status. Note: the Actions available if you select a Status are not normally required. For example Change Push Status is a rare requirement, it should only be needed when there’s been a problem and OCLC Support have advised - e.g. the records were rejected due to some system wide issue that has now been resolved and therefore should be marked Ready To Update.) |
For an individual Title record, the Audit trail sheet is also populated with changes due to the export status:
Titles: Audit Trail
| Changed Date | Changed By | Changed Field | Old Value | New Value |
|---|---|---|---|---|
| 18-FEB-2015 | Sys Admin | -PUBD @ EDS Export/ftp | RTP @ EDS Export/ftp | |
| 18-FEB-2015 | Sys Admin | RTP @ EDS Export/ftp | -PUBD @ EDS Export/ftp |
NOTE: The update process is fully automatic. If a catalogue record is changed, OLIB will automatically include the changed record in the Update file when it is next scheduled to export updates.
The Audit Trail sheet on the catalogue record will show changes to the export status. For example, RTP (Titles with this status are due to have an update sent to the external system during the next daystart run.)
Title records for deletion
- Delete the Title record as normal. OLIB knows if a title has previously been sent to a 3rd party. It will therefore prevent the Title from being deleted.
- In this case you put the record in the Titles for Auto-Deletion folder.
It is anticipated that a Title might be modified such that it is no longer in the Title Set. This can occur when, for example, there are no more copies at the location(s) speficied for that Title Set. When this happens, OLIB will send a deletion to any associated 3rd paty systems.
When supplying a deletion to the 3rd party, OLIB will set the record's MARC Leader 05 Record Status to "d"(deleted), informing the 3rd party of the deletion. This is set in the data being sent, not in the OLIB database. This allows records to be sent for deletion to one 3rd party system without being deleted frrom all 3rd party systems with which this OLIB database is synchronising its data.
When daystart checks the Titles for Auto-Deletion folder, it will first process any appropriate deletions for 3rd party systems.