This can be related to memory issues on the machine OCR is being performed on. Try performing OCR on smaller batches of files or on a machine with more memory. If problems persist, contact OCLC Support with the CONTENTdm server address, the collection name and some sample files.
There is more on generating transcripts with OCR.