Data exchange

Find an overview of the data exchange features available in OLIB Service Pack 12.

This section includes a summary of the various enhancements to the import and export processes that are included in OLIB Service Pack 12.

WorldCat data sync enhancements

When using the WorldCat data sync process, “response” files are produced that contain information about the results of the data sync. Improvements have now been made to cater for large response files of several thousand records. A new field has also been added to make it easier to view the files.

.

The new attribute - Discovery System Report - in the Titles domain allows you to view the response details.

This shows each line of the report from data sync of the current Title record you are viewing:

This list attribute shows the date and time of the processing of the response, together with the OCLC Number, the severity and message information.

The new list attribute needs to be added to the Titles layout using Layout Manager. It is suggested that the attribute is placed close to the OPAC Status information.

Here it is on the OPAC sheet of the new RDA layout:

To view the WorldCat responses

  1. Go to Titles and retrieve the relevant record.
  2. In the sheet containing your OPAC information, review the responses in the Discovery System Report field.
  3. Make changes to the Title record if required, depending on the severity / message.
  4. When you tick the box to select a response, you can use the Delete link for that line. This allows you to remove each line once you have corrected it in the catalogue record.

Bibliographic Exception Detail file

This contains the local system control number and the OCLC control number of records that OCLC was unable to match to records in WorldCat

Format of file name: metacoll.[user-symbol].datasync.[Collection ID].D[date processed].T[time procssed].bibdetailexcpt.1.mrc.txt

Example file

name: metacoll.OCWMS.datasync.1002106.D20160729.T082400.bibdetailexcpt.1.mrc.txt

The bibdetailexcpt file includes Severe (logical), Critical (structural) and Minor responses. Titles that have either Severe or Critical responses will be marked Rejected.

During the processing, titles may be marked as Rejected for the following additional reasons:

  • More than one OCLC Number has been supplied for a single title*
  • The same OCLC Number has been supplied for multiple titles
  • The OCLC Number supplied is already assigned to another title
  • The title has been archived

In order to prevent the incorrect update of a record to a new status when processing a bibdetailexcpt response file several days after the file has been generated, the status will not be updated if it was changed more than 2 days** after the generation of the bibdetailexcpt file, unless the update is simply to “Sent to WorldCat, Holdings Pending”. This will ensure that records which have been corrected, re-sent (and possibly received, holdings updated and processing completed) are not rejected due to an error that has been remedied.

  • Processing and validation in this respect will occur only within blocks of 1,000 lines of the response file. If a subsequent OCLC Number for the same title appears much later in the bibdetailexcpt file, this will not be reported as an error.

** 2 days is interpreted as follows:

  • The date in the response file includes no time element and is therefore treated as midnight (00:00 hours)
  • Daystart may have sent the file at, for example, 01:30 BST, and the response file may be generated, for example, 90 minutes later at 03:00 BST
  • The data sync processing will therefore report this as 22:00 EDT the previous day, without the time element
  • The date and time of the current status of the title (the date and time it was sent by daystart) is compared with midnight 2 days after 00:00 on the date in the bibdetailexcpt file in order to determine whether or not to update the status.

Author Matching during import

During OLSTF and MARC imports, OLIB attempts to match on the author records.

The author matching process helps to achieve accurate author matching during imports and to prevent duplicate author records if possible for libraries who do not use the initials field in the Authors table. The process is the same for OLSTF and MARC imports.

During normal usage, you do not normally need to visit or maintain this information. However, it could be useful to ascertain why an imported record has mismatched an existing author. You can then amend the match value to avoid future mismatches.

Author initials generation

During the import, Author initials are now not generated if:

Match values

During imports, OLIB generates and maintains a list of match values against each author. You can view these on the Names layout introduced in OLIB 9 Service Pack 12:

Import Matching fields:

 

During the import

OLIB's author matching follows this process during imports:

  1. The author matching process first checks to see if ORCid, VIAF or ISNI are present on the incoming record. If one of these match an existing author, any further processing is ignored
  2. If not present, the import will generate a Match Value from the incoming data and attempt to match this with a Match Value for an existing author.
  3. If no match is found by Match Value, the import will attempt to match on the surname, forename, initials etc... in the author record itself
    • If a match is found, the process adds Match Values to the author it has matched with
    • If no match is found, the process creates a new author record in OLIB, with the generated Match Value
  4. If this causes a duplicate record, you can edit the Match Values in the Authority Control Details above for your preferred author record.

View a match value

From the Import Matching field, you can view any existing match values - if there has been a previous import since OLIB 9 Service Pack 12.

  1. Go to Cataloguing> Names and search for the author.
  2. Go to Authority Control Details> Import Matching field. For the current author, this will show information such as:

For any future imports, the Match Value is the value that will be used in preference for matching. The import processes will first attempt to match exactly and then attempt a case insensitive match on just the letters, numbers and spaces in the field.

You can see who the new value was generated by and when. In the Source field where an OCLC#, Import RCN or Batch number is shown, this match value was generated by an import.

Value generated by existing author record

If the import author matching does not find a successful match for an author, then OLIB matches on a value generated from the existing author record. To view what this would be:

  1. Go to Cataloguing> Names and search for the author.
  2. On the Authority Control Details sheet > Import Matching click New. OLIB generates a match value based on the author record:

  1. You can manually edit the match value in the list box, or it can be removed if you check the box and click the Delete link below the list. You can click New to create an additional match value. (The import process will try to match with the first value and if not an exact match, then try the next value).

Note

For MARC imports, the match value will be generated from all but the Relator Term and Relator Code subfields. The subfield codes other than $a will be included in the match value.

For OLSTF imports, the match value will be generated as Forenames Initials Surname for personal names, as seen when generated by New above. In addition, there is an OLSTF import setting that permits the dates associated with the author to be ignored as part of the matching. If this is then enabled, the search will be carried out as a secondary match after the generation of the match value with dates, and a match value including dates will be added to the new or existing author record.

WorldCat data sync communications

The WorldCat metadata API process now records its communications to aid diagnosis of problems.

This is recorded in the worldcat_exchange_diagnostics table. Note this is for OCLC staff Support usage only.

To improve diagnostics, the following additional changes were made:

​​​​​​​

“WorldCat Batchload” changed to “WorldCat data sync”

References to WorldCat Batchload have been corrected so that they now refer to WorldCat data sync.

For example –

MARC Export handling of invalid characters

The MARC export no longer exports a record if the MARC record it generates includes critical invalid characters.

This was required because WorldCat data sync was rejecting the whole file if the first record (and only the first record) had invalid characters in Leader/06 and Leader/07.

The following data elements are now validated to ensure that they only contain a valid character:

In addition, if a title record does not have a language of cataloguing, the default cataloguing language specified in Admin Client Defaults will be exported as the language of cataloguing.

New OLSTF Tags

Several new tags have been added for Export.

Tag

Data table

Description

PCM

Titles non-repeatable block

The title’s primary classmark

CP_GPTI

CP_GPSUBTI

Copies export table

The title record's grandparent title and subtitle

CP_GEO_COLL

CP_GEO_COLLD

Copies export table

Geographic Collection code and description

(copies.geo_collection)

A Titles OLSTF import tag has been added for the Collection Code in the copy record. The OLSTF export process has also been enhanced to support this tag.

Tag

Data table

Description

COLLN

(Copies repeatable block in the Titles import)

A Collection code. This OLSTF tag is for import and export

The COLLN Collection code field is available in the Copies domain via Layout Manager. The attribute is

Collection (for layouts)co_collection