Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite #33

Merged
merged 39 commits into from
Oct 23, 2022
Merged
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
99e370e
update requirements
kba Dec 28, 2020
5a9ea7d
ocrd-tool: add parameter description (placeholders)
kba Dec 29, 2020
07fd913
[wip] upgrade to use 3.x baseline segmenter
kba Dec 29, 2020
c530706
blla segmenter basically working
kba Dec 30, 2020
ef40b8e
segment: more robust handling of lines not matched to regions
kba Dec 30, 2020
33e765f
implement legacy segmenter
kba Dec 30, 2020
5feb52c
decouple CLIs, add -recognize
kba Dec 30, 2020
28ef7cb
implement basic recognition
kba Dec 30, 2020
b67e7c3
recognize: output words and glyphs with conf
kba Dec 31, 2020
276f542
ci: do not build python 3.5 anymore
kba Dec 31, 2020
c2ceef9
require core 2.22.0b3+ for kraken resources
kba Dec 31, 2020
8be2abd
calculate word coords from glyph coords
kba Jan 1, 2021
8d93794
segment: convert coords back to absolute
bertsky Jan 8, 2021
232a055
segment: enlarge regions to check for contained lines, ensure polygon…
bertsky Jan 8, 2021
97ee297
Update ocrd_kraken/recognize.py
kba Jan 21, 2021
2f2a2c6
recognize: handle whitespace at BOL
kba Jan 22, 2021
38e9a7b
ocrd_tool.json: add model content type
bertsky Feb 10, 2022
f4a0caf
improve ocrd-tool.json, depend on released kraken > 3
kba Feb 18, 2022
cb719e7
Update ocrd_kraken/ocrd-tool.json
kba Mar 7, 2022
9c052df
recognize: fix decoding in case word is empty
bertsky Jul 13, 2022
b85e147
segment: use binarized input only if use_legacy
bertsky Jul 13, 2022
bd4609c
segment: decode arbitrary region types, add param blla_classes
bertsky Jul 13, 2022
5b6d79c
segment: move model init to setup fn
bertsky Jul 13, 2022
0ff4dd7
recognize: use binarized image only if model expects it
bertsky Jul 13, 2022
bfc675d
recognize: move model init to setup fn
bertsky Jul 13, 2022
5c7000a
recognize: use proper relative coordinate system for words and glyphs
bertsky Jul 13, 2022
0eecf6c
Revert "segment: use binarized input only if use_legacy"
bertsky Jul 15, 2022
104f138
Revert "Revert "segment: use binarized input only if use_legacy""
bertsky Jul 18, 2022
8b57e46
fix typos
kba Jul 22, 2022
d27f1a4
fix test_binarize
kba Jul 22, 2022
6f4a7bf
ci: download model first
kba Jul 22, 2022
f8235c4
:package: v0.2.0
kba Jul 22, 2022
ad8eb16
recognize: clip bboxes to canvas
bertsky Oct 13, 2022
36a463d
ocrd-tool: add model resources
bertsky Oct 13, 2022
6297b33
CI: add Py39, fix Py36
bertsky Oct 13, 2022
446af81
udpate+fix Docker build
bertsky Oct 13, 2022
c7b7f18
documentation
bertsky Oct 13, 2022
45143de
Makefile: fix typo
bertsky Oct 13, 2022
831e017
CI: remove assets
bertsky Oct 13, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 6 additions & 9 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,14 @@ version: 2

jobs:

build-python35:
docker:
- image: python:3.5.10-buster
steps:
- checkout
- run: make deps-ubuntu deps-test deps install assets
- run: make test

build-python36:
docker:
- image: python:3.6.12-buster
steps:
- checkout
- run: make deps-ubuntu deps-test deps install assets
- run: ocrd resmgr download ocrd-kraken-segment blla.mlmodel
- run: ocrd resmgr download ocrd-kraken-recognize en_best.mlmodel
- run: make test

build-python37:
Expand All @@ -24,6 +18,8 @@ jobs:
steps:
- checkout
- run: make deps-ubuntu deps-test deps install assets
- run: ocrd resmgr download ocrd-kraken-segment blla.mlmodel
- run: ocrd resmgr download ocrd-kraken-recognize en_best.mlmodel
- run: make test

build-python38:
Expand All @@ -32,13 +28,14 @@ jobs:
steps:
- checkout
- run: make deps-ubuntu deps-test deps install assets
- run: ocrd resmgr download ocrd-kraken-segment blla.mlmodel
- run: ocrd resmgr download ocrd-kraken-recognize en_best.mlmodel
- run: make test

workflows:
version: 2
build:
jobs:
- build-python35
- build-python36
- build-python37
- build-python38
10 changes: 9 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,14 @@ Versioned according to [Semantic Versioning](http://semver.org/).

## Unreleased

## [0.1.2] - 2019-09-24
## [0.2.0] - 2022-07-18

Added:

* text recognition with `ocrd-kraken-recognize`, #33
* rewrite of `ocrd-kraken-segment`, #33

## [0.1.2] - 2020-09-24

Fixed:

Expand Down Expand Up @@ -43,6 +50,7 @@ Changed:
Initial release

<!-- link-labels -->
[0.2.0]: v0.2.0...v0.1.2
[0.1.2]: v0.1.2...v0.1.1
[0.1.1]: v0.1.1...v0.1.0
[0.1.0]: v0.1.0...v0.0.2
Expand Down
3 changes: 0 additions & 3 deletions ocrd_kraken/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +0,0 @@
from .binarize import KrakenBinarize
from .ocr import KrakenOcr
from .segment import KrakenSegment
5 changes: 0 additions & 5 deletions ocrd_kraken/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,9 @@

from ocrd.decorators import ocrd_cli_options, ocrd_cli_wrap_processor
from ocrd_kraken.binarize import KrakenBinarize
from ocrd_kraken.segment import KrakenSegment

@click.command()
@ocrd_cli_options
def ocrd_kraken_binarize(*args, **kwargs):
return ocrd_cli_wrap_processor(KrakenBinarize, *args, **kwargs)

@click.command()
@ocrd_cli_options
def ocrd_kraken_segment(*args, **kwargs):
return ocrd_cli_wrap_processor(KrakenSegment, *args, **kwargs)
Empty file added ocrd_kraken/cli/__init__.py
Empty file.
9 changes: 9 additions & 0 deletions ocrd_kraken/cli/binarize.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
import click

from ocrd.decorators import ocrd_cli_options, ocrd_cli_wrap_processor
from ocrd_kraken.binarize import KrakenBinarize

@click.command()
@ocrd_cli_options
def cli(*args, **kwargs):
return ocrd_cli_wrap_processor(KrakenBinarize, *args, **kwargs)
11 changes: 11 additions & 0 deletions ocrd_kraken/cli/recognize.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
import click

from ocrd.decorators import ocrd_cli_options, ocrd_cli_wrap_processor
from ocrd_kraken.recognize import KrakenRecognize

@click.command()
@ocrd_cli_options
def cli(*args, **kwargs):
return ocrd_cli_wrap_processor(KrakenRecognize, *args, **kwargs)


8 changes: 8 additions & 0 deletions ocrd_kraken/cli/segment.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
import click
from ocrd.decorators import ocrd_cli_options, ocrd_cli_wrap_processor
from ocrd_kraken.segment import KrakenSegment

@click.command()
@ocrd_cli_options
def cli(*args, **kwargs):
return ocrd_cli_wrap_processor(KrakenSegment, *args, **kwargs)
5 changes: 3 additions & 2 deletions ocrd_kraken/config.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import json
from pkg_resources import resource_string
from pkg_resources import resource_filename

OCRD_TOOL = json.loads(resource_string(__name__, 'ocrd-tool.json').decode('utf8'))
with open(resource_filename(__name__, 'ocrd-tool.json'), 'r', encoding='utf-8') as f:
OCRD_TOOL = json.load(f)
44 changes: 0 additions & 44 deletions ocrd_kraken/ocr.py

This file was deleted.

106 changes: 82 additions & 24 deletions ocrd_kraken/ocrd-tool.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"git_url": "https://github.com/OCR-D/ocrd_kraken",
"version": "0.0.2",
"version": "0.2.0",
"tools": {
"ocrd-kraken-binarize": {
"executable": "ocrd-kraken-binarize",
Expand All @@ -15,6 +15,7 @@
"description": "Binarize images with kraken",
"parameters": {
"level-of-operation": {
"description": "level-of-operation",
"type": "string",
"default": "page",
"enum": ["page", "block", "line"]
Expand All @@ -32,43 +33,100 @@
"description": "Block segmentation with kraken",
"parameters": {
"text_direction": {
"type": "string",
"description": "Sets principal text direction",
"enum": ["horizontal-lr", "horizontal-rl", "vertical-lr", "vertical-rl"],
"type": "string",
"description": "Sets principal text direction",
"enum": ["horizontal-lr", "horizontal-rl", "vertical-lr", "vertical-rl"],
"default": "horizontal-lr"
},
"script_detect": {
"type": "boolean",
"description": "Enable script detection on segmenter output",
"maxcolseps": {
"description": "Maximum number of column separators. Set to 0 for single-column text to avoid unnecessary computation.",
"type": "number",
"format": "integer",
"default": 2
},
"scale": {
"description": "mean xheight size of glyphs (guessed if zero)",
"type": "number",
"format": "float",
"default": 0
},
"black_colseps": {
"description": "Whether column separators are assumed to be vertical black lines or not",
"type": "boolean",
"default": false
},
"maxcolseps": {"type": "number", "format": "integer", "default": 2},
"scale": {"type": "number", "format": "float", "default": 0},
"black_colseps": {"type": "boolean", "default": false},
"white_colseps": {"type": "boolean", "default": false},
"level-of-operation": {
"remove_hlines": {
"description": "Remove horizontal colseps before segmentation",
"type": "boolean",
"default": true
},
"blla_model": {
"description": "Model used for baseline detection and page segmentation. Ignored if use_legacy.",
"type": "string",
"default": "page",
"enum": ["page", "block", "line"]
"format": "uri",
"content-type": "application/python-cpickle",
"cacheable": true,
"default": "blla.mlmodel"
},
"blla_classes": {
"description": "Class mapping for the region types trained into blla_model.",
"type": "object",
"minProperties": 2,
"additionalProperties": { "type": "string",
"enum": ["TextRegion", "ImageRegion", "LineDrawingRegion",
"GraphicRegion", "TableRegion", "ChartRegion",
"MapRegion", "SeparatorRegion", "MathsRegion",
"ChemRegion", "MusicRegion", "AdvertRegion",
"NoiseRegion", "UnknownRegion", "CustomRegion"] },
"default": {"text": "TextRegion", "image": "ImageRegion", "line drawing": "LineDrawingRegion",
"graphic": "GraphicRegion", "table": "TableRegion", "chart": "ChartRegion",
"map": "MapRegion", "separator": "SeparatorRegion", "maths": "MathsRegion",
"chem": "ChemRegion", "music": "MusicRegion", "advert": "AdvertRegion",
"noise": "NoiseRegion", "unknown": "UnknownRegion", "custom": "CustomRegion"}
},
"device": {
"description": "GPU ID or 'cpu' to run on CPU only",
"type": "string",
"default": "cpu"
},
"use_legacy": {
"description": "Use legacy box segmenter as opposed to neural net baseline segmenter",
"type": "boolean",
"default": false
}
}
},
"ocrd-kraken-ocr": {
"executable": "ocrd-kraken-ocr",
"ocrd-kraken-recognize": {
"executable": "ocrd-kraken-recognize",
"categories": ["Text recognition and optimization"],
"steps": [
"recognition/text-recognition"
],
"steps": ["recognition/text-recognition"],
"description": "OCR with kraken",
"parameters": {
"lines-json": {
"model": {
"description": "OCR model to recognize with",
"type": "string",
"format": "uri",
kba marked this conversation as resolved.
Show resolved Hide resolved
"content-type": "application/python-cpickle",
"cacheable": true,
"default": "en_best.mlmodel"
},
"pad": {
"description": "Extra blank padding to the left and right of text line.",
"type": "number",
"format": "integer",
"default": 16
},
"bidi_reordering": {
"description": "Reorder classes in the ocr_record according to the Unicode bidirectional algorithm for correct display.",
"type": "boolean",
"default": true
},
"device": {
"description": "CUDA ID (e.g. 'cuda:0') for computation on GPU, or 'cpu' to run on CPU only",
"type": "string",
"format": "url",
"required": "true",
"description": "URL to line segmentation in JSON"
"default": "cpu"
}
}
}

}
}
Loading