by Kurt Breish, Chief Technology Officer, nextScan, Inc.
Abstract
An innovative film conversion processing approach , called Ribbon Scanning, is presented. This revolutionary approach to film and fiche scanning reduces conversion costs by optimizing equipment utilization and operator productivity. Ribbon Scanning digitizes an entire roll of film or jacket of fiche from top-to-bottom and end-to-end in grayscale, and stores it as a single file called the Ribbon File. Ribbon Scanning has been incorporated into the latest film processing software called NextStar. NextStar features, combined with the Ribbon Scanning approach, solve most of the problems resulting from density and filming related issues. It eliminates the need for rescans. NextStar is modular and expandable, and the software components communicate between multiple platforms allowing work to be scheduled and shared between many operators.
Executive Summary
NextStar software, the latest development by nextScan, introduces an innovative film processing methodology called Ribbon Scanning. With Ribbon Scanning, an entire roll of film or jacket of fiche is digitized from top-to-bottom and from end-to-end in grayscale, then stored as a single Ribbon file. NextStar with Ribbon Scanning was developed to solve most of the challenges encountered today in the conversion process from microfilm and microfiche to digital images, and to maximize the equipment utilization and operator’s productivity. NextStar allows the user to verify that all images were properly captured and identifies any image detection or density problems, then it allows the operator to correct those issues in a post-scan audit. All image data is captured initially in grayscale, then any images that need attention are highlighted. Any special image enhancements can be performed at the audit station, enabling optimum scan time and speed, optimum quality of output, and accuracy of the overall process. NextStar eliminates the need for rescans, delivers unmatched image quality, and outputs images that actually match your database. NextStar enables the user to manage end-to-end film conversion process and it is modular and expandable. From basic configurations, to large distributed production systems, the software components communicate between multiple platforms so that work can be scheduled and shared between many operators, even operators located in remote locations.
Ribbon Scanning
NextStar software digitizes rollfilm or microfiche using a new innovative concept called Ribbon Scanning. In this process, an entire roll of film or microfiche is digitized from top-to-bottom and end to-end in grayscale and stored as a single ribbon file.
This new method of scanning, Ribbon Scanning, was developed to solve many of the problems seen today in the conversion processes from microfilm and microfiche to digital images. One of the main characteristics of Ribbon Scanning is the ability to verify that all images were properly digitized, and, if any image detection or density problems are identified, NextStar allows the operator to correct those issues in a post-scan audit environment, eliminating the need for rescans. Not only does this approach yield higher utilization of the scanner and optimum operator productivity, without any complicated setup involved, but allows the scanner to run at rated speed. NextStar also leads to a much more accurate conversion process, with far superior image quality over a wider range of density and filming related issues. At last, the output images will actually match your database.
NextStar Workflow
NextStar software manages your entire end-to-end conversion process. It is modular and expandable – from a basic configuration installation where all components run on the nextScan scanners, to large distributed production systems where the software components communicate between multiple platforms, work is scheduled and shared between many operators. In this environment, tasks are coordinated by a central workflow controller and database. NextStar also offers a remote scan capability by allowing scanning to disk-packs and then sneaker net (or Express Delivery) the Ribbon data to your centralized processing facility.
The following picture depicts a conceptual flow diagram of the basic Ribbon Scanning process.
Processing Overview
Scanning
The NextStar scanning process consists of Scan software that runs at the scanner and CreateHierarchy software that executes where the Ribbon data is stored to disk. CreateHierarchy can be installed as a standalone program, or can be configured to run in the background as a service. The scanner captures the entire rollfilm or microfiche as a single ribbon image and sends it to the CreateHierarchy program in real time via a TCP/IP socket. The Scan application pays no attention to the framing requirements, file formats or image processing. In traditional Scan applications, the scan application must find the individual images, and must apply the image processing necessary to do enhancement, cropping, deskew, speckle removal, etc., during the actual scan.
These traditional processes limit the speed of the scanner in high resolution or large format scanning jobs. Since NextStar is not performing these tasks at scan time, the scanner is only limited by the network bandwidth and true scanner speed (camera capture speed). For example, if a scanner is scanning duplex rollfilm at 48x reduction and 200 DPI, the scanner will be required to process about 96 Mbytes/sec of image data. Typical scan applications on high performance PC’s can only process about 40 Mbytes/sec of image data with the above mentioned processing load. This means your expensive scanner is likely running at less than half of its rated capacity. However with NextStar, there is no processing being performed during the scanning phase, so you are only limited by the network bandwidth. In the case of NextStar on a 1Gig Ethernet link, that is about 100 Mbytes/second, so your nextScan scanner is performing as expected and then your investment in a high speed scanner is justified indeed.
The NextStar Capture software digitizes rollfilm and microfiche at high-speed and creates a single grayscale ribbon file from the data captured. Rollfilm is captured top to bottom, and microfiche images are captured in entire rows defined by the user.
Once the scanning process starts, the lamp and gamma correction values automatically adjust to the optimal level necessary (based on the film’s image density) to achieve the best image quality. The lamp and gamma correction values can also be adjusted manually.
Capture is easy to use and configure, requiring little training to learn how to perform production scanning.
CreateHierarchy
When CreateHierarchy software is installed, it is installed by default as a service that starts when the computer is turned on, and runs in the background.
CreateHierarchy is the other end of the scan connection and operates in real time as the scanner captures the data. CreateHierarchy software receives the scanned image data from the NextStar Scan application, and has the responsibility for writing the actual Ribbon data to disk storage. In the case of the FlexScan scanner, it is a local process on the scanner and writes the Ribbon to two local Raid drives in the scanner. With the Eclipse scanner, a remote server handles capturing the data that it sent to it via a 1Gig Ethernet dedicated link. Additionally, with Eclipse, the possibility exists to use external disk-packs to store the Ribbons and then send them to a centralized processing facility, where they are mounted on a NextStar Ribbon processing unit.
Detect
The NextStar Detect software runs automatically after CreateHierarchy and locates all the frames in the Ribbon. The setup and configuration of Detect is also available from the audit workstation and those “config” files can then be saved for detection on future Ribbons. There are two different frame detection methods; NS1 and NS2. To optimize processing options, It is intended to offer in the future more Detect versions of software from nextScan and from third party software developers. These methods have been developed with different approaches to solve specific issues. Both are relatively automatic, and require little intervention to correctly locate images. NS1 is designed to handle a large range of non-blipped film with complex image content, and NS2 is designed for automatic format recognition. NS2 can determine, among other features, if the roll has blips, which side of the film the blips are on, and if it is duplex. NS2 also handles skew detection.
Auditor
The NextStar Auditor software provides a means of performing quality assurance on all Ribbon files created by the NextStar system. The Auditor is the user interface to the Ribbon data and has a complete suite of tools for performing quality control over Ribbons scanned.
The Auditor gives you the ability to correct for any detection problems (such as missed or overlapped frames), mistakes, make image adjustments, insert or delete images and the ability to get the best image quality possible with the flexibility to output the images in a number of formats and indexing schemes. It can also be used to configure and run the desired frame detection parameters, configure and run output options, and configure image quality enhancements.
The Auditor also allows for a view of the entire rollfilm or microfiche, including all data in between frames. This is where the verification that all images have been found and image quality and readability is verified. Once the Audit process is complete, the XML data file is updated with the needed corrections and the file is marked ready for Output.The following picture depicts the Hardware components and software that runs on each step
Editing and Quality Control
The NextStar Auditor software includes the following features:
- Detect is a separate process accessed from the Auditor and is responsible for locating all the individual images in a particular Ribbon. The Auditor has two different detect options, and additionally supports third party detection software. The Detect process creates a separate XML data file that contains all the location and skew information of the individual images that is used by the Auditor Software and Auditor Output processes.
- A blackout setting, to quickly define images that were detected but not captured
- Quality control to fine-tune detection, make corrections, delete files (singly or in groups), or apply reverse numbering
- For microfiche: Allows grid templates for segmenting Grid/COM/Step and Repeat type formats – easily allows switching between different types of grids
- Intelligent detection algorithms with multi-Ribbon level options for best results, including algorithms written to custom-detect specific types of film formats (such as small frame gaps, duplex or blipped microfilm)
- Automatic horizontal and vertical detection algorithms
- Full scroll through entire roll, quickly or complete microfiche view
- Horizontal/vertical splitting
- Allows operator to easily select the order of available Ribbons for processing through specific filters. i.e. server, drives, Job ID or manually via browser
Output Options
The NextStar Auditor software is easy to learn and use. It includes functionality for setting output preferences.
Auditor Output option features include:
- Customized output file format and indexing preferences including multi-page plus multi-level blipped
- Image Defining: You can define images with the best image quality settings; apply processing and output file format settings over individual, global, and groups of images
- Automatic thresholding – automatic contrast adjustment to auto compensate contrast over variable densities within the microfilm
- Customized processing options. i.e. extract, crop, mirroring, invert, scaling, rotate and blip setup
- Export and import saved settings (an advantage for production environments)
Output
The Output reads in the XML data file and the Ribbon data, extracts the individual images, and outputs them according to the XML data that was specified by the Detect and Audit processes. It is the interface to the outside world so-to-speak. Output should be run on a server with a connection to a network that will receive the final individual images. Normally Output is run on the same server that CreateHierarchy is run on, and those servers have two NIC’s (network interface controllers), one for the scan side of the network and one for the customer network to output the images to.
Controller
The Controller keeps track of the Ribbon files, always keeping track of the status: items that have been processed, and whether these items have been written to disk.
From the Controller, it can be determined the status of each item, and can be managed the processing of each Ribbon forward as needed. It also has a state management and report generation capabilities.
Third Party Add-ins
nextScan has developed a NextStar SOK for 3rd party software developers to create additional workflow software. This includes detection software, OCR/ICR packages, bar code reading etc. This software is scheduled by the Controller just as the NextStar components are, and the results can be stored in the XML data file along with any NextStar data.