More General OCR

[KannamSridharKumar]

On the example image, More General OCR at the bottom,
music notes, chemical compound, some geometrical shapes are shown.

Whats the python command to extract such things?
I've tried all the example code provided, but they are extracting as plain text only.

Thanks,

[Ucas-HaoranWei]

--type format is OK.

[KannamSridharKumar]

I tried that, it didn't work. It didn't extract chemical formula.
Can you pls share the full command. Thanks,

[Ucas-HaoranWei]

The full command is the same as 'format' OCR.
Can you show me your input image？

[KannamSridharKumar]

I couldn't install it on colab from source code, so I'm using via HuggingFace Pipeline.

res = model.chat(tokenizer, image_file, ocr_type='format')

https://huggingface.co/stepfun-ai/GOT-OCR2_0

I've also tried the HF demo - https://huggingface.co/spaces/ucaslcl/GOT_online

[Ucas-HaoranWei]

I know, it is because the difference between this image and the images we rendered is too large.
There are two solutions:

1. Use the chemical compound like this image:
   
2. Fine-tune the model with your data.
   Thank you～

[KannamSridharKumar]

Thank you very much, I understand, can you please share the 3 example images used in the More General OCR section. I tried with musical notes and geometry images but it didn't work but probably my images are too different from whats the model has been trained on.

[Ucas-HaoranWei]

Hi, the benchmark.zip includes samples.