Machine Learning for Archival Image Classification

by Ismail Msuya

I believe that the adoption of Artificial Intelligence in libraries and information environments would open the door to transformative opportunities for increasing library usage and enhancing the overall user experiences. A prime example of such opportunities includes the implementation of visual sensors that utilize AI technologies in a library setting so that patrons would be able to self-checkout a book by simply taking a book off the shelf. The detection of what book was taken by whom and the association between that information into the Integrated Library System would be established autonomously as they walk out of the library. Another example of a transformative opportunity would be using AI and visual sensors to translate sign language into text or speech at the circulation and reference desks.

The 2021 IDEA Institute on Artificial Intelligence has broadened my understanding of Artificial Intelligence and how machine learning (ML) techniques could be implemented in a library setting. This led me to learn more about existing computer vision algorithms and deep neural networks such as OpenCV, PyTorch, and Tensorflow that could be implemented for image classification, object detection, and recognition. Use of DialogFlow-a natural language understanding platform used to design and integrate a conversational user interface into mobile apps, web applications, devices, bots, interactive voice response systems, and related uses—to implement a chat agent that could interface with our Space Reservation system allowing patrons to reserve spaces directly from the chatbox. AI project planning and design, AI ethics, becoming thoughtful and distance our models from discriminatory design when implementing AI solutions.

One of the AI projects that I am excited to pursue is an idea that was brought to me by a colleague. The idea is to use Machine Learning techniques to autonomously identify a specific group of persons from a collection of photographs, sort the images with the identified persons into separate folders and modify the metadata for those images to have the persons’ names as subjects. It would simplify the overall identification and retrieval process. As a result, anyone researching the history around these persons would benefit from improved subject access to images, as would university offices that request images with specific people or activities in them for publicity purposes.

Part of the design process is to gain clarity on what problem this Machine Learning model would be addressing, and identify ethical guidelines for a project like this. Research not only on computer vision techniques for image processing, analysis, and recognition but also, the current image identification workflow and how the end-user would be affected. Additionally, I’d need to identify whether the images are in analog, digital format, or both, as well as how many diverse collections of these images we have to train the model. It’s important to have answers to these questions to gain more clarity and identify any biases that could be passed on to the model. I am hoping that this project will be the beginning. After this, I will train the model to find other identifiable objects in photographs and videos. For example, what university sports uniforms look like, allowing it to pick out images of people playing those sports.