Recognizing coins with artificial intelligence

Over the last year, many new opportunities became available for me to learn about coins with the use of artificial intelligence. These developments truly open a new chapter in what coin collecting could mean to me without vision.

Previously I wrote a lot about recognizing coins with technology, so far what is still working and available is Seeing AI by Microsoft. What I was able to do is using optical character recognition to get enough text from a coin to tell what it is. It mostly works with good quality coins and good quality pictures. After I determine what a coin is, I use Numista to get more information, however, what I am able to get really depends on the descriptions users provide.

These days, there are several options to use AI to describe images. The other advantage here is that unlike with regular character recognition, AI understands when the text goes around the coin, and instead of looking for straight lines of characters, it is able to process the characters in a circular direction. Thus, the accuracy greatly improves.

But here comes the interesting part, when I get a coin described, I can ask questions about it and get information about certain details. The downside is, since most of these services use ChatGPT, it does make some assumptions, and also makes up some information. This is improving weekly, but at this point, I cannot rely on the description to a hundred percent. But with the usage of other online resources, I can minimize the errors.

The way this works is instead of taking a picture of a coin, I scan it. The photo quality may not be the best numismatic photography, but the results are rather consistent. Once I scan the coin, I have it set to transfer to my phone automatically, where I can use the Be My Eyes app, which recently added AI photo recognition. I also tried to upload pictures to ChatGPT directly on the desktop, but there I am getting worse results.

Once the photo is on my phone, I can run the Be My Eyes application, which describes the coin, and then I can ask questions. The entire process takes about a half minute. For each question, I usually get a response in about 5 seconds after that.

Let’s look at a few examples:

First I scanned the bicentennial half Dollar with the Independence hall. I resized the picture below to speed up the page, but originally I worked with a high resolution picture, I set the scanning quality as high as my scanner allowed. This is what I got from AI. I will not change anything in the responses, I will copy and paste them exactly as I got them.

The image is of a coin with the following details:

– The text “UNITED STATES OF AMERICA” at the top edge

– A depiction of Independence Hall with a large clock tower in the center

– The year “200 YEARS OF FREEDOM” with “1776-1976” beneath the building

– The Latin phrase “E PLURIBUS UNUM” to the right of the building

– The word “INDEPENDENCE HALL” below the building

– The denomination “HALF DOLLAR” at the bottom edge

This is great, much better than what I could get last year, namely, nothing.

Descriptions and the wording can greatly vary at different times, or by using different applications, but the end result is about the same.

Now let’s ask a question:

What does the building look like? Explain it in great detail.

The building depicted is a two-story structure with a central clock tower rising above the main roof. The details are as follows:

– The central clock tower is prominent, with a small dome and weathervane at the very top.

– The clock face is visible on the tower, just below the dome.

– The tower has several small rectangular windows, with a larger arched window below the clock.

– The main building has a symmetrical design, with two wings flanking the central tower.

– Each wing has two levels of windows; the first floor has tall, narrow windows, and the second floor has shorter windows.

– The roof of each wing appears to be flat with a balustrade along the edge.

– The central section of the building, just below the tower, has an arched entranceway.

– There are decorative elements on the building, such as a pediment above the entrance and ornate detailing around the windows and along the roofline.

Now this is amazing. But let’s dive in further.

What does the clock look like?

The clock on the tower is round with a clearly defined border. The clock face is marked with Roman numerals for the hours. The hands of the clock are not distinctly visible in the image. The clock is set into the tower just below the smaller arched window at the top of the tower.

What is interesting here is that when I got this described earlier, using the same image, I even asked what time does the clock show, and it told me 4:10. So, at this point, something is incorrect. It probably made up something earlier, and I don’t have a good way to verify what the clock is showing, I asked my wife to check it with a magnifier and she couldn’t figure it out.

One thing is interesting though, if you upload a good quality picture, it can probably tell you more than what one can see with a naked eye, because it is able to analyze the image.

So, I will leave it up to you, if you can see the image above, to decide how accurate it is. Knowing the coin, I think it is at least very close.

Let’s look at another coin, this is also going to be a building, the Hungarian National Bank on the 1992 200 Forint.

This is the initial description I get:

The image shows a coin lying on a grey fabric surface. The coin has text at the top that reads “MAGYAR NEMZETI BANK” which suggests it is Hungarian currency. Below the text, there is an image of a building with a classical facade, featuring columns and a central pediment. Beneath the building, there is a signature that reads “Bod Val” and the date “1992”. The coin appears to be silver in color and is slightly worn.

Based on the text, AI guesses that it is Hungarian. Though it is an excellent guess, this is the type of information I don’t need. In other circumstances, such approach could be useful when I try to understand my environment, but when analyzing coins, I would like to get the facts and nothing else, but at this point it is not possible. Instead, I can ask all the questions that will give me the answers I need.

Describe the building in great detail.

The building depicted on the coin has a neoclassical architectural style. It features a central section with a triangular pediment supported by a series of evenly spaced columns. The columns appear to be of the Corinthian order, characterized by ornate capitals. Flanking the central section are two symmetrical wings with a flat roofline, each with two levels of windows. The windows on the upper level are shorter and aligned with the columns, while the lower level windows are taller. The building’s facade has a strong sense of symmetry and grandeur, typical of neoclassical design.

How many columns are there?

There are six columns visible on the building depicted on the coin.

And so on, we could continue probing the system and describe the details of parts of the building, the coin as a whole, the style of the text, anything.

Instead of diving into more information here, I would like to show you another approach I like to take once a coin is recognized, because I can be more certain that I get more accurate information. I don’t have a good way to tell how good are the pictures I took or scanned. Therefore, once I know which coins I would like to get described, I go over to Numista, because there is a good chance that somebody took a better picture, and hopefully of a better quality coin.

I would be interested to learn what Queen Elizabeth looks like. I chose a Canadian 25 Cent piece, designed by Dora de Pédery-hunt.

I bring up the Numista page with the coin, and let AI do the work. Here is the initial description:

The image shows two sides of a Canadian 25-cent coin.

On one side:

– There’s a profile portrait of Queen Elizabeth II facing right.

– Above her is the text “ELIZABETH II D.G. REGINA”.

– Below is the text “1867-1992” indicating a commemorative year.

On the other side:

– There’s an image of a caribou head facing left.

– Above the caribou, the text reads “CANADA”.

– Below the caribou, it says “25 CENTS”.

– The coin’s edge has a dotted border on both sides.

Describe the queen in extremely fine detail.

The portrait of Queen Elizabeth II shows her in profile from the shoulders up. She has a mature appearance with her hair styled in elegant curls that are tucked under a tiara. The tiara has intricate detailing and several peaks. Her ear is partially visible under her hair, adorned with a pearl earring. She is wearing a necklace with visible detailed links and a pendant, which appears to be a pearl as well. Her facial features are composed, with a straight nose, closed lips, and a pronounced jawline. Her eye is depicted with a heavy eyelid and she has a gentle but regal expression. The Queen’s attire, though not fully visible, suggests a formal garment with a high neckline.

I specifically asked to describe her in extremely fine detail, because there is a limit to descriptions, and unless I ask, I don’t get the great details. There is a particular character limit, so if I need even more detail, I need to have certain aspects described. Which makes sense, because if I just would like to know what I’m holding in my hand, I don’t need pages upon pages of description. However, if I want to research a coin that’s exactly what I need. So, one more question:

Describe her crown in great detail.

The tiara on Queen Elizabeth II’s head is detailed and ornate. It features several arches with a prominent peak at the center. Each arch is embellished with what appears to be gemstones and several spherical elements, giving the impression of a jeweled crown with intricate metalwork. The central peak is taller than the surrounding arches and is capped with a cross-like design, suggesting a traditional regal element. The tiara’s base sits closely to the Queen’s head, encircling her forehead and disappearing into her hairline at the sides. The overall design of the tiara is elegant and stately, befitting a monarch.

This is amazing. For those of you who can see and have interacted with coins for decades may not be much. But for me, it is like being in the toy store. And this is only getting better. Just a few months ago I wanted to write this article and decided not to, because AI made up so much stuff. Today, it is pretty usable. Not to mention, it also took me some time to learn how to ask good questions. It seems that the more specifics I ask, the less it makes up. Ok, except the time on the clock.

If I was publishing anything more complex than a blog post, I would still probably have the accuracy verified. This information is not ready to build a research on, but for me as a collector it is extremely helpful. Particularly because there is only so much description you will get from Numista. Of course, when Numista has detailed descriptions, it makes the catalog more searchable, but after all, people are expected to use the descriptions together with the photos, and as you know, a picture is worth a thousand words. And now, it is for me, too, because I can get a thousand words out of it.