Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
56dd4c7ef4 | ||
|
|
924596e38d | ||
|
|
2ea0b717f9 | ||
|
|
38f8ca76a6 | ||
|
|
ac49ad1bcf | ||
|
|
d74bf1eaa9 | ||
|
|
cf3c89bf25 | ||
|
|
b491cd2181 | ||
|
|
70144c8fe1 | ||
|
|
0d832bf7fa |
No files matched your search
@@ -2,8 +2,5 @@ language: python
|
|||||||
python:
|
python:
|
||||||
- "3.6.4"
|
- "3.6.4"
|
||||||
- "3.7"
|
- "3.7"
|
||||||
install:
|
|
||||||
- pip install pycryptodome
|
|
||||||
- pip install chardet
|
|
||||||
script:
|
script:
|
||||||
nosetests --nologcapture
|
nosetests --nologcapture
|
||||||
@@ -1,11 +1,36 @@
|
|||||||
# PDFMajor
|
# PDFMajor
|
||||||
|
|
||||||
|
<table>
|
||||||
|
<tr>
|
||||||
|
<td>Latest Release</td>
|
||||||
|
<td>
|
||||||
|
<a href="https://pypi.org/project/pdfmajor/">
|
||||||
|
<img src="https://img.shields.io/pypi/v/pdfmajor.svg" alt="latest release" />
|
||||||
|
</a>
|
||||||
|
</td>
|
||||||
|
</tr>
|
||||||
|
</table>
|
||||||
|
|
||||||
PDF-Major is a fork of `PDFMiner.six`. It is meant to be a more light-weight implementation that makes fewer assumptions on the data. Additionally, this version intends to be more up to date with recent versions of python and make liberal use of the typing capabilities of python for improved future maintainability.
|
PDF-Major is a fork of `PDFMiner.six`. It is meant to be a more light-weight implementation that makes fewer assumptions on the data. Additionally, this version intends to be more up to date with recent versions of python and make liberal use of the typing capabilities of python for improved future maintainability.
|
||||||
|
|
||||||
|
The ultimate goal is to have a functional-programming with typing fork of the original library.
|
||||||
|
|
||||||
## Documentation
|
## Documentation
|
||||||
|
|
||||||
See documentation [here](https://asosnovsky.github.io/pdfmajor/).
|
See documentation [here](https://asosnovsky.github.io/pdfmajor/).
|
||||||
|
|
||||||
|
## Basic Example
|
||||||
|
```py
|
||||||
|
from pdfmajor.extractor import extract_items_from_pdf
|
||||||
|
|
||||||
|
for page in extract_items_from_pdf('path/to/your/file.pdf'):
|
||||||
|
print('page-start', page)
|
||||||
|
for item in page:
|
||||||
|
print(' ', item)
|
||||||
|
print('page-end', page)
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
## Why Another Fork?
|
## Why Another Fork?
|
||||||
|
|
||||||
`PDFMiner` was designed to run in python 2.7>, `PDFMiner.six` was written to bring cross-version support for python 2 and 3. This versions intends to make use of the latest improvement in the python language while remaining fairly simple to develop with.
|
`PDFMiner` was designed to run in python 2.7>, `PDFMiner.six` was written to bring cross-version support for python 2 and 3. This versions intends to make use of the latest improvement in the python language while remaining fairly simple to develop with.
|
||||||
@@ -13,6 +38,7 @@ See documentation [here](https://asosnovsky.github.io/pdfmajor/).
|
|||||||
|
|
||||||
## Features
|
## Features
|
||||||
|
|
||||||
|
* Functional extraction method based on generators
|
||||||
* Written entirely in Python.
|
* Written entirely in Python.
|
||||||
* Parse, analyze, and convert PDF documents.
|
* Parse, analyze, and convert PDF documents.
|
||||||
* PDF-1.7 specification support. (well, almost)
|
* PDF-1.7 specification support. (well, almost)
|
||||||
|
|||||||
+57
-1
@@ -2,11 +2,44 @@
|
|||||||
|
|
||||||
The library is constructed from 4 base modules:
|
The library is constructed from 4 base modules:
|
||||||
|
|
||||||
|
* **extractor**: contains a high-level function for functional extraction of pdf-assets
|
||||||
* **interpreter**: contains low-level classes for extracting fundamental data-structures from the documents
|
* **interpreter**: contains low-level classes for extracting fundamental data-structures from the documents
|
||||||
* **converters**: contains high-level classes for conversion of the fundamental pdf structures to other formats
|
* **converters**: contains high-level classes for conversion of the fundamental pdf structures to other formats
|
||||||
* **imagewriter**: contains a simple implementation for converting PDF Image Streams to png/bmp/img formats
|
* **imagewriter**: contains a simple implementation for converting PDF Image Streams to png/bmp/img formats
|
||||||
* **layouts**: contains an abstraction layer for the data-structures found in the pdf
|
* **layouts**: contains an abstraction layer for the data-structures found in the pdf
|
||||||
|
|
||||||
|
## extractors
|
||||||
|
Contains a high-level function for functional extraction of pdf-assets
|
||||||
|
|
||||||
|
### Example
|
||||||
|
```py
|
||||||
|
from pdfmajor.extractor import extract_items_from_pdf
|
||||||
|
|
||||||
|
for page in extract_items_from_pdf('path/to/your/file.pdf'):
|
||||||
|
print('page-start', page)
|
||||||
|
for item in page:
|
||||||
|
print(' ', item)
|
||||||
|
print('page-end', page)
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
### extract_items_from_pdf(input_file_path,maxpages,password,caching,check_extractable,pagenos,debug_level) -> generator[LTPage]
|
||||||
|
|
||||||
|
This generator-function yields individual pages which contain their respected items.
|
||||||
|
|
||||||
|
#### Arguments
|
||||||
|
|
||||||
|
- `input_file_path`: [str](str)
|
||||||
|
- `maxpages`: [int](#) defaults to 0
|
||||||
|
- `password`: [str](#) defaults to None
|
||||||
|
- `caching`: [bool](#) defaults to True
|
||||||
|
- `check_extractable`: [bool](#) defaults to True
|
||||||
|
- `pagenos`: [List[int]](#) defaults to None
|
||||||
|
- `debug_level`: [logging.levels](#https://docs.python.org/3/library/logging.html#levels) defaults logging.WARNING
|
||||||
|
|
||||||
|
#### Return Value
|
||||||
|
This function returns a generator that makes `LTPage`
|
||||||
|
|
||||||
## interpreter
|
## interpreter
|
||||||
WIP
|
WIP
|
||||||
|
|
||||||
@@ -100,4 +133,27 @@ WIP
|
|||||||
|
|
||||||
## layouts
|
## layouts
|
||||||
|
|
||||||
WIP
|
WIP: this documentation set is not yet complete.
|
||||||
|
|
||||||
|
### Layout Items
|
||||||
|
|
||||||
|
All layout items extend the `LTItem` class. There are two kinds of layout items:
|
||||||
|
|
||||||
|
- LTComponent: extends the base `LTItem` class, this class will have additional values such as boundary boxes, height and width
|
||||||
|
- LTContainer: extends the `LTComponent` class, this class is used to contain elements of the pdf that would have child elements. Iterating on this element will output its child elements.
|
||||||
|
|
||||||
|
|
||||||
|
#### Layout Containers
|
||||||
|
All of these classes extend the LTContainer class.
|
||||||
|
|
||||||
|
- LTPage: a layout item containing all elements of the given page.
|
||||||
|
- LTFigure: a layout item containing LTCurves and LTImages
|
||||||
|
- LTCharBlock: a layout item containing LTChars
|
||||||
|
|
||||||
|
#### Layout Components
|
||||||
|
All of these classes extend the LTComponent class.
|
||||||
|
|
||||||
|
- LTChar: an individual character
|
||||||
|
- LTCurves: represents a collection of svg-paths (available under `self.paths`)
|
||||||
|
- LTImage: a component containing information regarding an image
|
||||||
|
|
||||||
+27
-1
@@ -1,11 +1,36 @@
|
|||||||
# PDFMajor
|
# PDFMajor
|
||||||
|
|
||||||
|
<table>
|
||||||
|
<tr>
|
||||||
|
<td>Latest Release</td>
|
||||||
|
<td>
|
||||||
|
<a href="https://pypi.org/project/pdfmajor/">
|
||||||
|
<img src="https://img.shields.io/pypi/v/pdfmajor.svg" alt="latest release" />
|
||||||
|
</a>
|
||||||
|
</td>
|
||||||
|
</tr>
|
||||||
|
</table>
|
||||||
|
|
||||||
PDF-Major is a fork of `PDFMiner.six`. It is meant to be a more light-weight implementation that makes fewer assumptions on the data. Additionally, this version intends to be more up to date with recent versions of python and make liberal use of the typing capabilities of python for improved future maintainability.
|
PDF-Major is a fork of `PDFMiner.six`. It is meant to be a more light-weight implementation that makes fewer assumptions on the data. Additionally, this version intends to be more up to date with recent versions of python and make liberal use of the typing capabilities of python for improved future maintainability.
|
||||||
|
|
||||||
|
The ultimate goal is to have a functional-programming with typing fork of the original library.
|
||||||
|
|
||||||
## Documentation
|
## Documentation
|
||||||
|
|
||||||
See documentation [here](https://asosnovsky.github.io/pdfmajor/).
|
See documentation [here](https://asosnovsky.github.io/pdfmajor/).
|
||||||
|
|
||||||
|
## Basic Example
|
||||||
|
```py
|
||||||
|
from pdfmajor.extractor import extract_items_from_pdf
|
||||||
|
|
||||||
|
for page in extract_items_from_pdf('path/to/your/file.pdf'):
|
||||||
|
print('page-start', page)
|
||||||
|
for item in page:
|
||||||
|
print(' ', item)
|
||||||
|
print('page-end', page)
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
## Why Another Fork?
|
## Why Another Fork?
|
||||||
|
|
||||||
`PDFMiner` was designed to run in python 2.7>, `PDFMiner.six` was written to bring cross-version support for python 2 and 3. This versions intends to make use of the latest improvement in the python language while remaining fairly simple to develop with.
|
`PDFMiner` was designed to run in python 2.7>, `PDFMiner.six` was written to bring cross-version support for python 2 and 3. This versions intends to make use of the latest improvement in the python language while remaining fairly simple to develop with.
|
||||||
@@ -13,6 +38,7 @@ See documentation [here](https://asosnovsky.github.io/pdfmajor/).
|
|||||||
|
|
||||||
## Features
|
## Features
|
||||||
|
|
||||||
|
* Functional extraction method based on generators
|
||||||
* Written entirely in Python.
|
* Written entirely in Python.
|
||||||
* Parse, analyze, and convert PDF documents.
|
* Parse, analyze, and convert PDF documents.
|
||||||
* PDF-1.7 specification support. (well, almost)
|
* PDF-1.7 specification support. (well, almost)
|
||||||
|
|||||||
+14
-3
@@ -53,7 +53,7 @@
|
|||||||
<a class="current" href=".">Home</a>
|
<a class="current" href=".">Home</a>
|
||||||
<ul class="subnav">
|
<ul class="subnav">
|
||||||
|
|
||||||
<li class="toctree-l2"><a href="#pdf-major">PDF-Major</a></li>
|
<li class="toctree-l2"><a href="#pdfmajor">PDFMajor</a></li>
|
||||||
|
|
||||||
<ul>
|
<ul>
|
||||||
|
|
||||||
@@ -110,7 +110,18 @@
|
|||||||
<div role="main">
|
<div role="main">
|
||||||
<div class="section">
|
<div class="section">
|
||||||
|
|
||||||
<h1 id="pdf-major">PDF-Major</h1>
|
<h1 id="pdfmajor">PDFMajor</h1>
|
||||||
|
<table>
|
||||||
|
<tr>
|
||||||
|
<td>Latest Release</td>
|
||||||
|
<td>
|
||||||
|
<a href="https://pypi.org/project/pdfmajor/">
|
||||||
|
<img src="https://img.shields.io/pypi/v/pdfmajor.svg" alt="latest release" />
|
||||||
|
</a>
|
||||||
|
</td>
|
||||||
|
</tr>
|
||||||
|
</table>
|
||||||
|
|
||||||
<p>PDF-Major is a fork of <code>PDFMiner.six</code>. It is meant to be a more light-weight implementation that makes fewer assumptions on the data. Additionally, this version intends to be more up to date with recent versions of python and make liberal use of the typing capabilities of python for improved future maintainability.</p>
|
<p>PDF-Major is a fork of <code>PDFMiner.six</code>. It is meant to be a more light-weight implementation that makes fewer assumptions on the data. Additionally, this version intends to be more up to date with recent versions of python and make liberal use of the typing capabilities of python for improved future maintainability.</p>
|
||||||
<h2 id="documentation">Documentation</h2>
|
<h2 id="documentation">Documentation</h2>
|
||||||
<p>See documentation <a href="https://asosnovsky.github.io/pdfmajor/">here</a>.</p>
|
<p>See documentation <a href="https://asosnovsky.github.io/pdfmajor/">here</a>.</p>
|
||||||
@@ -216,5 +227,5 @@ SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.</p>
|
|||||||
|
|
||||||
<!--
|
<!--
|
||||||
MkDocs version : 1.0.4
|
MkDocs version : 1.0.4
|
||||||
Build Date UTC : 2018-10-12 22:22:34
|
Build Date UTC : 2018-11-04 21:27:38
|
||||||
-->
|
-->
|
||||||
@@ -1 +1 @@
|
|||||||
{"config":{"lang":["en"],"prebuild_index":false,"separator":"[\\s\\-]+"},"docs":[{"location":"","text":"PDF-Major PDF-Major is a fork of PDFMiner.six . It is meant to be a more light-weight implementation that makes fewer assumptions on the data. Additionally, this version intends to be more up to date with recent versions of python and make liberal use of the typing capabilities of python for improved future maintainability. Documentation See documentation here . Why Another Fork? PDFMiner was designed to run in python 2.7>, PDFMiner.six was written to bring cross-version support for python 2 and 3. This versions intends to make use of the latest improvement in the python language while remaining fairly simple to develop with. Features Written entirely in Python. Parse, analyze, and convert PDF documents. PDF-1.7 specification support. (well, almost) CJK languages and vertical writing scripts support. Various font types (Type1, TrueType, Type3, and CID) support. Basic encryption (RC4) support. Font-Color extraction Shape fill and stroke color extraction Note : We took out the layout-analysis process in this version (there is no more LTTextHorizontal or LTTextVertical). While the mathematics behind the grouping process was sound, the coupling of the layout-analysis process with the parsing and interpretation process produced unfriendly-code. This feature could be brought back by extending the pdfmajor.layouts.PDFLayoutAnalyzer class, but at the current time is not supported. How to Install Source Install Python 3.6.4 or newer. clone this repo git clone https://github.com/asosnovsky/pdfmajor * install repo python setup.py install Pypi Install Python 3.6.4 or newer. install repo pip install pdfmajor Terms and Conditions (This is so-called MIT/X License) Copyright (c) 2018-2019 Ariel Sosnovsky Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.","title":"Home"},{"location":"#pdf-major","text":"PDF-Major is a fork of PDFMiner.six . It is meant to be a more light-weight implementation that makes fewer assumptions on the data. Additionally, this version intends to be more up to date with recent versions of python and make liberal use of the typing capabilities of python for improved future maintainability.","title":"PDF-Major"},{"location":"#documentation","text":"See documentation here .","title":"Documentation"},{"location":"#why-another-fork","text":"PDFMiner was designed to run in python 2.7>, PDFMiner.six was written to bring cross-version support for python 2 and 3. This versions intends to make use of the latest improvement in the python language while remaining fairly simple to develop with.","title":"Why Another Fork?"},{"location":"#features","text":"Written entirely in Python. Parse, analyze, and convert PDF documents. PDF-1.7 specification support. (well, almost) CJK languages and vertical writing scripts support. Various font types (Type1, TrueType, Type3, and CID) support. Basic encryption (RC4) support. Font-Color extraction Shape fill and stroke color extraction Note : We took out the layout-analysis process in this version (there is no more LTTextHorizontal or LTTextVertical). While the mathematics behind the grouping process was sound, the coupling of the layout-analysis process with the parsing and interpretation process produced unfriendly-code. This feature could be brought back by extending the pdfmajor.layouts.PDFLayoutAnalyzer class, but at the current time is not supported.","title":"Features"},{"location":"#how-to-install","text":"","title":"How to Install"},{"location":"#source","text":"Install Python 3.6.4 or newer. clone this repo git clone https://github.com/asosnovsky/pdfmajor * install repo python setup.py install","title":"Source"},{"location":"#pypi","text":"Install Python 3.6.4 or newer. install repo pip install pdfmajor","title":"Pypi"},{"location":"#terms-and-conditions","text":"(This is so-called MIT/X License) Copyright (c) 2018-2019 Ariel Sosnovsky Permission is hereby granted, free of charge, to any person obtaining aLine truncated
|
{"config":{"lang":["en"],"prebuild_index":false,"separator":"[\\s\\-]+"},"docs":[{"location":"","text":"PDFMajor Latest Release PDF-Major is a fork of PDFMiner.six . It is meant to be a more light-weight implementation that makes fewer assumptions on the data. Additionally, this version intends to be more up to date with recent versions of python and make liberal use of the typing capabilities of python for improved future maintainability. Documentation See documentation here . Why Another Fork? PDFMiner was designed to run in python 2.7>, PDFMiner.six was written to bring cross-version support for python 2 and 3. This versions intends to make use of the latest improvement in the python language while remaining fairly simple to develop with. Features Written entirely in Python. Parse, analyze, and convert PDF documents. PDF-1.7 specification support. (well, almost) CJK languages and vertical writing scripts support. Various font types (Type1, TrueType, Type3, and CID) support. Basic encryption (RC4) support. Font-Color extraction Shape fill and stroke color extraction Note : We took out the layout-analysis process in this version (there is no more LTTextHorizontal or LTTextVertical). While the mathematics behind the grouping process was sound, the coupling of the layout-analysis process with the parsing and interpretation process produced unfriendly-code. This feature could be brought back by extending the pdfmajor.layouts.PDFLayoutAnalyzer class, but at the current time is not supported. How to Install Source Install Python 3.6.4 or newer. clone this repo git clone https://github.com/asosnovsky/pdfmajor * install repo python setup.py install Pypi Install Python 3.6.4 or newer. install repo pip install pdfmajor Terms and Conditions (This is so-called MIT/X License) Copyright (c) 2018-2019 Ariel Sosnovsky Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.","title":"Home"},{"location":"#pdfmajor","text":"Latest Release PDF-Major is a fork of PDFMiner.six . It is meant to be a more light-weight implementation that makes fewer assumptions on the data. Additionally, this version intends to be more up to date with recent versions of python and make liberal use of the typing capabilities of python for improved future maintainability.","title":"PDFMajor"},{"location":"#documentation","text":"See documentation here .","title":"Documentation"},{"location":"#why-another-fork","text":"PDFMiner was designed to run in python 2.7>, PDFMiner.six was written to bring cross-version support for python 2 and 3. This versions intends to make use of the latest improvement in the python language while remaining fairly simple to develop with.","title":"Why Another Fork?"},{"location":"#features","text":"Written entirely in Python. Parse, analyze, and convert PDF documents. PDF-1.7 specification support. (well, almost) CJK languages and vertical writing scripts support. Various font types (Type1, TrueType, Type3, and CID) support. Basic encryption (RC4) support. Font-Color extraction Shape fill and stroke color extraction Note : We took out the layout-analysis process in this version (there is no more LTTextHorizontal or LTTextVertical). While the mathematics behind the grouping process was sound, the coupling of the layout-analysis process with the parsing and interpretation process produced unfriendly-code. This feature could be brought back by extending the pdfmajor.layouts.PDFLayoutAnalyzer class, but at the current time is not supported.","title":"Features"},{"location":"#how-to-install","text":"","title":"How to Install"},{"location":"#source","text":"Install Python 3.6.4 or newer. clone this repo git clone https://github.com/asosnovsky/pdfmajor * install repo python setup.py install","title":"Source"},{"location":"#pypi","text":"Install Python 3.6.4 or newer. install repo pip install pdfmajor","title":"Pypi"},{"location":"#terms-and-conditions","text":"(This is so-called MIT/X License) Copyright (c) 2018-2019 Ariel Sosnovsky Permission is hereby granted, free of chargeLine truncated
|
||||||
+2
-2
@@ -2,12 +2,12 @@
|
|||||||
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
|
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
|
||||||
<url>
|
<url>
|
||||||
<loc>None</loc>
|
<loc>None</loc>
|
||||||
<lastmod>2018-10-12</lastmod>
|
<lastmod>2018-11-04</lastmod>
|
||||||
<changefreq>daily</changefreq>
|
<changefreq>daily</changefreq>
|
||||||
</url>
|
</url>
|
||||||
<url>
|
<url>
|
||||||
<loc>None</loc>
|
<loc>None</loc>
|
||||||
<lastmod>2018-10-12</lastmod>
|
<lastmod>2018-11-04</lastmod>
|
||||||
<changefreq>daily</changefreq>
|
<changefreq>daily</changefreq>
|
||||||
</url>
|
</url>
|
||||||
</urlset>
|
</urlset>
|
||||||
Binary file not shown.
+28
-2
@@ -3,14 +3,39 @@
|
|||||||
# DO NOT EDIT THIS PAGE
|
# DO NOT EDIT THIS PAGE
|
||||||
#
|
#
|
||||||
"""
|
"""
|
||||||
# PDFMajor
|
# PDFMajor
|
||||||
|
|
||||||
|
<table>
|
||||||
|
<tr>
|
||||||
|
<td>Latest Release</td>
|
||||||
|
<td>
|
||||||
|
<a href="https://pypi.org/project/pdfmajor/">
|
||||||
|
<img src="https://img.shields.io/pypi/v/pdfmajor.svg" alt="latest release" />
|
||||||
|
</a>
|
||||||
|
</td>
|
||||||
|
</tr>
|
||||||
|
</table>
|
||||||
|
|
||||||
PDF-Major is a fork of `PDFMiner.six`. It is meant to be a more light-weight implementation that makes fewer assumptions on the data. Additionally, this version intends to be more up to date with recent versions of python and make liberal use of the typing capabilities of python for improved future maintainability.
|
PDF-Major is a fork of `PDFMiner.six`. It is meant to be a more light-weight implementation that makes fewer assumptions on the data. Additionally, this version intends to be more up to date with recent versions of python and make liberal use of the typing capabilities of python for improved future maintainability.
|
||||||
|
|
||||||
|
The ultimate goal is to have a functional-programming with typing fork of the original library.
|
||||||
|
|
||||||
## Documentation
|
## Documentation
|
||||||
|
|
||||||
See documentation [here](https://asosnovsky.github.io/pdfmajor/).
|
See documentation [here](https://asosnovsky.github.io/pdfmajor/).
|
||||||
|
|
||||||
|
## Basic Example
|
||||||
|
```py
|
||||||
|
from pdfmajor.extractor import extract_items_from_pdf
|
||||||
|
|
||||||
|
for page in extract_items_from_pdf('path/to/your/file.pdf'):
|
||||||
|
print('page-start', page)
|
||||||
|
for item in page:
|
||||||
|
print(' ', item)
|
||||||
|
print('page-end', page)
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
## Why Another Fork?
|
## Why Another Fork?
|
||||||
|
|
||||||
`PDFMiner` was designed to run in python 2.7>, `PDFMiner.six` was written to bring cross-version support for python 2 and 3. This versions intends to make use of the latest improvement in the python language while remaining fairly simple to develop with.
|
`PDFMiner` was designed to run in python 2.7>, `PDFMiner.six` was written to bring cross-version support for python 2 and 3. This versions intends to make use of the latest improvement in the python language while remaining fairly simple to develop with.
|
||||||
@@ -18,6 +43,7 @@ See documentation [here](https://asosnovsky.github.io/pdfmajor/).
|
|||||||
|
|
||||||
## Features
|
## Features
|
||||||
|
|
||||||
|
* Functional extraction method based on generators
|
||||||
* Written entirely in Python.
|
* Written entirely in Python.
|
||||||
* Parse, analyze, and convert PDF documents.
|
* Parse, analyze, and convert PDF documents.
|
||||||
* PDF-1.7 specification support. (well, almost)
|
* PDF-1.7 specification support. (well, almost)
|
||||||
@@ -75,7 +101,7 @@ OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
|
|||||||
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||||
|
|
||||||
"""
|
"""
|
||||||
__version__ = (1, 1, 5)
|
__version__ = (1, 2, 0)
|
||||||
|
|
||||||
if __name__ == '__main__':
|
if __name__ == '__main__':
|
||||||
print(
|
print(
|
||||||
|
|||||||
@@ -25,7 +25,7 @@ def get_color(col: PDFGraphicStateColor):
|
|||||||
elif color_type == 'cmyk':
|
elif color_type == 'cmyk':
|
||||||
return f"cmyk({','.join(map(str, color_val))})"
|
return f"cmyk({','.join(map(str, color_val))})"
|
||||||
elif color_type == 'gray':
|
elif color_type == 'gray':
|
||||||
return f'rgba(0,0,0,{1-color_val})'
|
return f'rgba(0,0,0,{1-color_val[0]})'
|
||||||
elif color_type == "custom":
|
elif color_type == "custom":
|
||||||
# Todo: convert these to more friendly color types
|
# Todo: convert these to more friendly color types
|
||||||
return f"custom({col.custom['type']})"
|
return f"custom({col.custom['type']})"
|
||||||
@@ -134,6 +134,8 @@ class HTMLConverter(PDFConverter):
|
|||||||
'position': 'absolute',
|
'position': 'absolute',
|
||||||
'left': f'{item.x0}px',
|
'left': f'{item.x0}px',
|
||||||
'bottom': f'{item.y0}px',
|
'bottom': f'{item.y0}px',
|
||||||
|
'width': f'{item.width}px',
|
||||||
|
'height': f'{item.height}px',
|
||||||
}
|
}
|
||||||
path = ""
|
path = ""
|
||||||
svg_attr = {
|
svg_attr = {
|
||||||
@@ -141,42 +143,36 @@ class HTMLConverter(PDFConverter):
|
|||||||
'z-index': 2
|
'z-index': 2
|
||||||
}
|
}
|
||||||
|
|
||||||
if item.height > 0:
|
p_vals = { "x": [], "y": [] }
|
||||||
dy = 1.0/item.height
|
|
||||||
dx = (item.width/item.height)/(item.x1-item.x0)
|
|
||||||
svg_attr.update({
|
|
||||||
"viewBox": "0 0 %s %s" % ( item.width/item.height, 1 )
|
|
||||||
})
|
|
||||||
else:
|
|
||||||
dx = 1.0/(item.x1-item.x0)
|
|
||||||
dy = 1.0
|
|
||||||
svg_attr.update({
|
|
||||||
"viewBox": "0 0 1 1"
|
|
||||||
})
|
|
||||||
|
|
||||||
if item.height > 1:
|
for p in item.paths:
|
||||||
svg_attr.update({
|
for pt in p.points:
|
||||||
'height': f'{item.height}px',
|
p_vals['x'].append(pt.x)
|
||||||
})
|
p_vals['y'].append(pt.y)
|
||||||
else:
|
|
||||||
svg_attr.update({
|
mx_x = max([*p_vals['x'], 1])
|
||||||
'height': '1px',
|
mn_x = min([*p_vals['x'], 0])
|
||||||
})
|
mx_y = max([*p_vals['y'], 1])
|
||||||
|
mn_y = min([*p_vals['y'], 0])
|
||||||
if item.width > 1:
|
|
||||||
svg_attr.update({
|
if mx_x == mn_x:
|
||||||
'width': f'{item.width}px',
|
mn_x -= 1
|
||||||
})
|
if mx_y == mn_y:
|
||||||
else:
|
mn_y -= 1
|
||||||
svg_attr.update({
|
|
||||||
'width': '1px',
|
dx = 1/( mx_x - mn_x )
|
||||||
})
|
dy = (item.height/item.width)/( mx_y - mn_y )
|
||||||
|
ROUND_VAL = 5
|
||||||
|
svg_attr.update({
|
||||||
|
# "viewBox": f"{min(p_vals['x'])} {min(p_vals['y'])} {max(p_vals['x'])} {max(p_vals['y'])}",
|
||||||
|
"viewBox": f"{round(mn_x*dx, ROUND_VAL)} {round(mn_y*dy, ROUND_VAL)} {round(mx_x*dx , ROUND_VAL)} {round(mx_y*dy, ROUND_VAL)}",
|
||||||
|
})
|
||||||
|
|
||||||
for p in item.paths:
|
for p in item.paths:
|
||||||
if p.method == CurvePath.METHOD.MOVE_TO:
|
if p.method == CurvePath.METHOD.MOVE_TO:
|
||||||
path += f'M{" ".join( f"{(item.width - (p.x-item.x0) )*dx} {(item.height- (p.y-item.y0) )*dy}" for p in p.points )} '
|
path += f'M{" ".join( f"{round(dx*pt.x, ROUND_VAL)} {round(dy*pt.y, ROUND_VAL)}" for pt in p.points )} '
|
||||||
if p.method == CurvePath.METHOD.LINE_TO:
|
if p.method == CurvePath.METHOD.LINE_TO:
|
||||||
path += f'L{" ".join( f"{(item.width - (p.x-item.x0) )*dx} {( item.height - (p.y-item.y0) )*dy}" for p in p.points )} '
|
path += f'L{" ".join( f"{round(dx*pt.x, ROUND_VAL)} {round(dy*pt.y, ROUND_VAL)}" for pt in p.points )} '
|
||||||
if p.method == CurvePath.METHOD.CLOSE_PATH:
|
if p.method == CurvePath.METHOD.CLOSE_PATH:
|
||||||
path += 'Z'
|
path += 'Z'
|
||||||
with self.place_elm_with_child('svg', svg_attr, css):
|
with self.place_elm_with_child('svg', svg_attr, css):
|
||||||
|
|||||||
@@ -1,17 +1,19 @@
|
|||||||
import logging
|
|
||||||
import re
|
import re
|
||||||
import os
|
import os
|
||||||
|
|
||||||
from typing import List
|
from typing import List
|
||||||
from io import BytesIO, StringIO, TextIOWrapper
|
from io import BytesIO, StringIO, TextIOWrapper
|
||||||
|
|
||||||
|
from tqdm import tqdm
|
||||||
|
|
||||||
from ..layouts import PDFLayoutAnalyzer
|
from ..layouts import PDFLayoutAnalyzer
|
||||||
from ..interpreter.PDFResourceManager import PDFResourceManager
|
from ..interpreter.PDFResourceManager import PDFResourceManager
|
||||||
from ..interpreter.PDFPage import PDFPage
|
from ..interpreter.PDFPage import PDFPage
|
||||||
from ..interpreter.PDFPageInterpreter import PDFPageInterpreter
|
from ..interpreter.PDFPageInterpreter import PDFPageInterpreter
|
||||||
from ..imagewriter import ImageWriter
|
from ..imagewriter import ImageWriter
|
||||||
|
from ..utils import set_log_level, get_logger, logging
|
||||||
|
|
||||||
log = logging.getLogger(__name__)
|
log = get_logger(__name__)
|
||||||
|
|
||||||
## PDFConverter
|
## PDFConverter
|
||||||
##
|
##
|
||||||
@@ -29,11 +31,10 @@ class PDFConverter(PDFLayoutAnalyzer):
|
|||||||
check_extractable: bool = True,
|
check_extractable: bool = True,
|
||||||
pagenos: List[int] = None,
|
pagenos: List[int] = None,
|
||||||
dont_export_images: bool = False,
|
dont_export_images: bool = False,
|
||||||
debug: bool = False,
|
debug_level: int = logging.WARNING,
|
||||||
) -> TextIOWrapper:
|
) -> TextIOWrapper:
|
||||||
|
|
||||||
if debug:
|
set_log_level(debug_level)
|
||||||
log.setLevel(logging.DEBUG)
|
|
||||||
|
|
||||||
if dont_export_images:
|
if dont_export_images:
|
||||||
image_folder_path = None
|
image_folder_path = None
|
||||||
@@ -53,9 +54,12 @@ class PDFConverter(PDFLayoutAnalyzer):
|
|||||||
interpreter = PDFPageInterpreter(rsrcmgr, device)
|
interpreter = PDFPageInterpreter(rsrcmgr, device)
|
||||||
log.debug("Reading pages....")
|
log.debug("Reading pages....")
|
||||||
pages = PDFPage.get_pages(input_file, pagenos=pagenos, maxpages=maxpages, password=password, caching=caching, check_extractable=check_extractable)
|
pages = PDFPage.get_pages(input_file, pagenos=pagenos, maxpages=maxpages, password=password, caching=caching, check_extractable=check_extractable)
|
||||||
for idx, page in enumerate(pages):
|
|
||||||
log.debug(f"Reading page #{idx}")
|
page_waiter = tqdm(enumerate(pages), disable=debug_level > logging.INFO)
|
||||||
interpreter.process_page(page)
|
for _, page in page_waiter:
|
||||||
|
page_waiter.set_description("Processing pdf...")
|
||||||
|
interpreter.process_page(page)
|
||||||
|
page_waiter.close()
|
||||||
device.close()
|
device.close()
|
||||||
return output_file
|
return output_file
|
||||||
|
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ import json
|
|||||||
|
|
||||||
from typing import Dict
|
from typing import Dict
|
||||||
|
|
||||||
|
from xml.sax import saxutils
|
||||||
from contextlib import contextmanager
|
from contextlib import contextmanager
|
||||||
|
|
||||||
from ..layouts import LTPage
|
from ..layouts import LTPage
|
||||||
@@ -32,7 +33,7 @@ def get_color(col: PDFGraphicStateColor):
|
|||||||
return f'gray({color_val})'
|
return f'gray({color_val})'
|
||||||
elif color_type == "custom":
|
elif color_type == "custom":
|
||||||
# Todo: convert these to more friendly color types
|
# Todo: convert these to more friendly color types
|
||||||
return f"custom({col.custom['type']})"
|
return f"{col.custom['type']}({col.custom['defintion']})"
|
||||||
return ""
|
return ""
|
||||||
|
|
||||||
FIRST_CAP_RE = re.compile('(.)([A-Z][a-z]+)')
|
FIRST_CAP_RE = re.compile('(.)([A-Z][a-z]+)')
|
||||||
@@ -82,7 +83,7 @@ class XMLConverter(PDFConverter):
|
|||||||
lineend = ''
|
lineend = ''
|
||||||
deep_space = ''
|
deep_space = ''
|
||||||
attrs = " ".join([
|
attrs = " ".join([
|
||||||
f'{name}="{value}"'
|
f'{name}="{saxutils.escape(str(value))}"'
|
||||||
for name, value in attr_dict.items()
|
for name, value in attr_dict.items()
|
||||||
])
|
])
|
||||||
self.write(f"<{tag_name} {attrs}>", lineend=lineend)
|
self.write(f"<{tag_name} {attrs}>", lineend=lineend)
|
||||||
@@ -93,7 +94,7 @@ class XMLConverter(PDFConverter):
|
|||||||
|
|
||||||
def place_elm_close(self, tag_name: str, attr_dict: Dict[str, str] = {}, as_childless = True):
|
def place_elm_close(self, tag_name: str, attr_dict: Dict[str, str] = {}, as_childless = True):
|
||||||
attrs = " ".join([
|
attrs = " ".join([
|
||||||
f'{name}="{value}"'
|
f'{name}="{saxutils.escape(str(value))}"'
|
||||||
for name, value in attr_dict.items()
|
for name, value in attr_dict.items()
|
||||||
])
|
])
|
||||||
if as_childless:
|
if as_childless:
|
||||||
@@ -130,7 +131,7 @@ class XMLConverter(PDFConverter):
|
|||||||
'y0': char.y0,
|
'y0': char.y0,
|
||||||
'y1': char.y1,
|
'y1': char.y1,
|
||||||
}, no_additional_char=True):
|
}, no_additional_char=True):
|
||||||
self.write(char.get_text(), lineend='', deep_space="")
|
self.write(saxutils.escape(char.get_text()), lineend='', deep_space="")
|
||||||
|
|
||||||
def render_char_block(self, char_block: LTCharBlock):
|
def render_char_block(self, char_block: LTCharBlock):
|
||||||
attr = {
|
attr = {
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ from .HTMLConverter import HTMLConverter
|
|||||||
from .JSONConverter import JSONConverter
|
from .JSONConverter import JSONConverter
|
||||||
from .XMLConverter import XMLConverter
|
from .XMLConverter import XMLConverter
|
||||||
from .TextConverter import TextConverter
|
from .TextConverter import TextConverter
|
||||||
|
from ..utils import logging
|
||||||
|
|
||||||
def convert_file(
|
def convert_file(
|
||||||
input_file: io.TextIOWrapper,
|
input_file: io.TextIOWrapper,
|
||||||
@@ -19,7 +20,8 @@ def convert_file(
|
|||||||
caching: bool = True,
|
caching: bool = True,
|
||||||
check_extractable: bool = True,
|
check_extractable: bool = True,
|
||||||
pagenos: List[int] = None,
|
pagenos: List[int] = None,
|
||||||
debug: bool = False,
|
dont_export_images: bool = False,
|
||||||
|
debug_level: int = logging.WARNING,
|
||||||
out_type: str = 'html',
|
out_type: str = 'html',
|
||||||
):
|
):
|
||||||
if output_file is None:
|
if output_file is None:
|
||||||
@@ -36,54 +38,58 @@ def convert_file(
|
|||||||
raise Exception("Please make sure that the file name and output type match!")
|
raise Exception("Please make sure that the file name and output type match!")
|
||||||
if out_type == 'html':
|
if out_type == 'html':
|
||||||
return HTMLConverter.parse_file(
|
return HTMLConverter.parse_file(
|
||||||
input_file,
|
input_file=input_file,
|
||||||
output_file,
|
output_file=output_file,
|
||||||
image_folder_path,
|
image_folder_path=image_folder_path,
|
||||||
codec,
|
codec=codec,
|
||||||
maxpages,
|
maxpages=maxpages,
|
||||||
password,
|
password=password,
|
||||||
caching,
|
caching=caching,
|
||||||
check_extractable,
|
check_extractable=check_extractable,
|
||||||
pagenos,
|
pagenos=pagenos,
|
||||||
debug,
|
dont_export_images=dont_export_images,
|
||||||
|
debug_level=debug_level,
|
||||||
)
|
)
|
||||||
elif out_type == 'xml':
|
elif out_type == 'xml':
|
||||||
return XMLConverter.parse_file(
|
return XMLConverter.parse_file(
|
||||||
input_file,
|
input_file=input_file,
|
||||||
output_file,
|
output_file=output_file,
|
||||||
image_folder_path,
|
image_folder_path=image_folder_path,
|
||||||
codec,
|
codec=codec,
|
||||||
maxpages,
|
maxpages=maxpages,
|
||||||
password,
|
password=password,
|
||||||
caching,
|
caching=caching,
|
||||||
check_extractable,
|
check_extractable=check_extractable,
|
||||||
pagenos,
|
pagenos=pagenos,
|
||||||
debug,
|
dont_export_images=dont_export_images,
|
||||||
|
debug_level=debug_level,
|
||||||
)
|
)
|
||||||
elif out_type == 'json':
|
elif out_type == 'json':
|
||||||
return JSONConverter.parse_file(
|
return JSONConverter.parse_file(
|
||||||
input_file,
|
input_file=input_file,
|
||||||
output_file,
|
output_file=output_file,
|
||||||
image_folder_path,
|
image_folder_path=image_folder_path,
|
||||||
codec,
|
codec=codec,
|
||||||
maxpages,
|
maxpages=maxpages,
|
||||||
password,
|
password=password,
|
||||||
caching,
|
caching=caching,
|
||||||
check_extractable,
|
check_extractable=check_extractable,
|
||||||
pagenos,
|
pagenos=pagenos,
|
||||||
debug,
|
dont_export_images=dont_export_images,
|
||||||
|
debug_level=debug_level,
|
||||||
)
|
)
|
||||||
elif out_type == 'text':
|
elif out_type == 'text':
|
||||||
return TextConverter.parse_file(
|
return TextConverter.parse_file(
|
||||||
input_file,
|
input_file=input_file,
|
||||||
output_file,
|
output_file=output_file,
|
||||||
image_folder_path,
|
image_folder_path=image_folder_path,
|
||||||
codec,
|
codec=codec,
|
||||||
maxpages,
|
maxpages=maxpages,
|
||||||
password,
|
password=password,
|
||||||
caching,
|
caching=caching,
|
||||||
check_extractable,
|
check_extractable=check_extractable,
|
||||||
pagenos,
|
pagenos=pagenos,
|
||||||
debug,
|
dont_export_images=dont_export_images,
|
||||||
|
debug_level=debug_level,
|
||||||
)
|
)
|
||||||
else: raise Exception("Please specify out_type as 'html' or 'xml' or 'json' or 'text'")
|
else: raise Exception("Please specify out_type as 'html' or 'xml' or 'json' or 'text'")
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
from ..utils import get_logger
|
||||||
|
|
||||||
|
log = get_logger(__name__)
|
||||||
|
|
||||||
|
from ..layouts import PDFLayoutAnalyzer, PDFPage, LTPage
|
||||||
|
|
||||||
|
class PDFExctractor(PDFLayoutAnalyzer):
|
||||||
|
def end_page(self, page: PDFPage):
|
||||||
|
assert not self._stack, str(len(self._stack))
|
||||||
|
assert isinstance(self.cur_item, LTPage), str(type(self.cur_item))
|
||||||
|
self.pageno += 1
|
||||||
|
return self.cur_item
|
||||||
@@ -0,0 +1,46 @@
|
|||||||
|
import re
|
||||||
|
import os
|
||||||
|
|
||||||
|
from typing import List
|
||||||
|
from io import BytesIO, StringIO, TextIOWrapper
|
||||||
|
|
||||||
|
from tqdm import tqdm
|
||||||
|
|
||||||
|
from ..interpreter.PDFResourceManager import PDFResourceManager
|
||||||
|
from ..interpreter.PDFPage import PDFPage
|
||||||
|
from ..interpreter.PDFPageInterpreter import PDFPageInterpreter
|
||||||
|
from ..imagewriter import ImageWriter
|
||||||
|
from ..utils import set_log_level, get_logger, logging
|
||||||
|
from .PDFExtractor import PDFExctractor
|
||||||
|
|
||||||
|
log = get_logger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
def extract_items_from_pdf(
|
||||||
|
input_file_path: str,
|
||||||
|
maxpages: int = 0,
|
||||||
|
password: str = None,
|
||||||
|
caching: bool = True,
|
||||||
|
check_extractable: bool = True,
|
||||||
|
pagenos: List[int] = None,
|
||||||
|
debug_level: int = logging.WARNING,
|
||||||
|
):
|
||||||
|
set_log_level(debug_level)
|
||||||
|
|
||||||
|
log.debug("Creating resources....")
|
||||||
|
rsrcmgr = PDFResourceManager(caching=caching)
|
||||||
|
device = PDFExctractor(
|
||||||
|
rsrcmgr=rsrcmgr,
|
||||||
|
pageno=0
|
||||||
|
)
|
||||||
|
interpreter = PDFPageInterpreter(rsrcmgr, device)
|
||||||
|
log.debug("Reading pages....")
|
||||||
|
with open(input_file_path, 'rb') as input_file:
|
||||||
|
pages = PDFPage.get_pages(input_file, pagenos=pagenos, maxpages=maxpages, password=password, caching=caching, check_extractable=check_extractable)
|
||||||
|
|
||||||
|
page_waiter = tqdm(enumerate(pages), disable=debug_level > logging.INFO)
|
||||||
|
for _, page in page_waiter:
|
||||||
|
page_waiter.set_description("Processing pdf...")
|
||||||
|
yield interpreter.process_page(page)
|
||||||
|
page_waiter.close()
|
||||||
|
device.close()
|
||||||
@@ -34,7 +34,7 @@ class PDFGraphicStateColor:
|
|||||||
|
|
||||||
@property
|
@property
|
||||||
def gray(self):
|
def gray(self):
|
||||||
if self.__gray is not None: return float(self.__gray)
|
if self.__gray is not None: return [float(self.__gray)]
|
||||||
else: raise self.NotSet
|
else: raise self.NotSet
|
||||||
|
|
||||||
@property
|
@property
|
||||||
|
|||||||
@@ -7,7 +7,9 @@ from .PDFStream import resolve1, int_value, list_value, dict_value
|
|||||||
from .PDFParser import PDFParser
|
from .PDFParser import PDFParser
|
||||||
from .PDFDocument import PDFDocument, PDFTextExtractionNotAllowed
|
from .PDFDocument import PDFDocument, PDFTextExtractionNotAllowed
|
||||||
|
|
||||||
log = logging.getLogger(__name__)
|
from ..utils import get_logger
|
||||||
|
|
||||||
|
log = get_logger(__name__)
|
||||||
|
|
||||||
# some predefined literals and keywords.
|
# some predefined literals and keywords.
|
||||||
LITERAL_PAGE = LIT('Page')
|
LITERAL_PAGE = LIT('Page')
|
||||||
@@ -89,7 +91,7 @@ class PDFPage(object):
|
|||||||
tree_type = tree.get('type')
|
tree_type = tree.get('type')
|
||||||
|
|
||||||
if tree_type is LITERAL_PAGES and 'Kids' in tree:
|
if tree_type is LITERAL_PAGES and 'Kids' in tree:
|
||||||
log.info('Pages: Kids=%r', tree['Kids'])
|
# log.info('Pages: Kids=%r', tree['Kids'])
|
||||||
for c in list_value(tree['Kids']):
|
for c in list_value(tree['Kids']):
|
||||||
for x in search(c, tree):
|
for x in search(c, tree):
|
||||||
yield x
|
yield x
|
||||||
|
|||||||
@@ -351,6 +351,24 @@ class PDFPageInterpreter(object):
|
|||||||
#self.do_cs(LITERAL_DEVICE_CMYK)
|
#self.do_cs(LITERAL_DEVICE_CMYK)
|
||||||
return
|
return
|
||||||
|
|
||||||
|
def __process_ICCBased(self, n: int, components: list):
|
||||||
|
if n == 1:
|
||||||
|
return {
|
||||||
|
"type": "gray",
|
||||||
|
"definition": components
|
||||||
|
}
|
||||||
|
elif n == 3:
|
||||||
|
return {
|
||||||
|
"type": "rgb",
|
||||||
|
"definition": components
|
||||||
|
}
|
||||||
|
elif n == 4:
|
||||||
|
return {
|
||||||
|
"type": "cmyk",
|
||||||
|
"definition": components
|
||||||
|
}
|
||||||
|
raise PDFInterpreterError("Unsupported ICCBased color")
|
||||||
|
|
||||||
# setcolor
|
# setcolor
|
||||||
def __process_scn(self):
|
def __process_scn(self):
|
||||||
if self.ncs:
|
if self.ncs:
|
||||||
@@ -359,8 +377,14 @@ class PDFPageInterpreter(object):
|
|||||||
raise PDFInterpreterError('No colorspace specified!')
|
raise PDFInterpreterError('No colorspace specified!')
|
||||||
components = self.pop(n)
|
components = self.pop(n)
|
||||||
obj_id = components[0]
|
obj_id = components[0]
|
||||||
|
if self.ncs.name == "ICCBased":
|
||||||
|
return self.__process_ICCBased(n, components)
|
||||||
if self.ncs.name not in self.resources.keys():
|
if self.ncs.name not in self.resources.keys():
|
||||||
raise PDFInterpreterError("Color space not in resources")
|
log.warning("Color space not in resources")
|
||||||
|
return {
|
||||||
|
"type": self.ncs.name,
|
||||||
|
"defintion": f"{n}|({','.join(map(str, components))})"
|
||||||
|
}
|
||||||
if obj_id.name not in self.resources[self.ncs.name].keys():
|
if obj_id.name not in self.resources[self.ncs.name].keys():
|
||||||
raise PDFInterpreterError("Invalid Color reference")
|
raise PDFInterpreterError("Invalid Color reference")
|
||||||
|
|
||||||
@@ -374,11 +398,27 @@ class PDFPageInterpreter(object):
|
|||||||
"defintion": color_data
|
"defintion": color_data
|
||||||
}
|
}
|
||||||
def do_SCN(self):
|
def do_SCN(self):
|
||||||
self.graphicstate.scolor.custom = self.__process_scn()
|
processed_col = self.__process_scn()
|
||||||
|
if processed_col['type'] == 'rgb':
|
||||||
|
self.graphicstate.scolor.rgb = processed_col['definition']
|
||||||
|
elif processed_col['type'] == 'gray':
|
||||||
|
self.graphicstate.scolor.gray = processed_col['definition'][0]
|
||||||
|
elif processed_col['type'] == 'cmyk':
|
||||||
|
self.graphicstate.scolor.cmyk = processed_col['definition']
|
||||||
|
else:
|
||||||
|
self.graphicstate.scolor.custom = processed_col
|
||||||
return
|
return
|
||||||
|
|
||||||
def do_scn(self):
|
def do_scn(self):
|
||||||
self.graphicstate.ncolor.custom = self.__process_scn()
|
processed_col = self.__process_scn()
|
||||||
|
if processed_col['type'] == 'rgb':
|
||||||
|
self.graphicstate.ncolor.rgb = processed_col['definition']
|
||||||
|
elif processed_col['type'] == 'gray':
|
||||||
|
self.graphicstate.ncolor.gray = processed_col['definition'][0]
|
||||||
|
elif processed_col['type'] == 'cmyk':
|
||||||
|
self.graphicstate.ncolor.cmyk = processed_col['definition']
|
||||||
|
else:
|
||||||
|
self.graphicstate.ncolor.custom = processed_col
|
||||||
return
|
return
|
||||||
|
|
||||||
def do_SC(self):
|
def do_SC(self):
|
||||||
@@ -584,8 +624,7 @@ class PDFPageInterpreter(object):
|
|||||||
ctm = (1, 0, 0, 1, -x0, -y0)
|
ctm = (1, 0, 0, 1, -x0, -y0)
|
||||||
self.device.begin_page(page, ctm)
|
self.device.begin_page(page, ctm)
|
||||||
self.render_contents(page.resources, page.contents, ctm=ctm)
|
self.render_contents(page.resources, page.contents, ctm=ctm)
|
||||||
self.device.end_page(page)
|
return self.device.end_page(page)
|
||||||
return
|
|
||||||
|
|
||||||
# render_contents(resources, streams, ctm)
|
# render_contents(resources, streams, ctm)
|
||||||
# Render the content streams.
|
# Render the content streams.
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
from typing import List
|
from typing import List
|
||||||
|
|
||||||
from ._base import LTComponent
|
from ._base import LTContainer
|
||||||
from .LTChar import LTChar
|
from .LTChar import LTChar
|
||||||
|
|
||||||
from ..utils import apply_matrix_norm, apply_matrix_pt, matrix2str, Bbox, Point
|
from ..utils import apply_matrix_norm, apply_matrix_pt, matrix2str, Bbox, Point
|
||||||
@@ -11,7 +11,7 @@ from ..interpreter.PDFTextState import PDFTextState
|
|||||||
|
|
||||||
## LTCharBlock
|
## LTCharBlock
|
||||||
##
|
##
|
||||||
class LTCharBlock(LTComponent):
|
class LTCharBlock(LTContainer):
|
||||||
|
|
||||||
def __init__(self,
|
def __init__(self,
|
||||||
chars: List[Point],
|
chars: List[Point],
|
||||||
@@ -20,7 +20,7 @@ class LTCharBlock(LTComponent):
|
|||||||
graphicstate: PDFGraphicState
|
graphicstate: PDFGraphicState
|
||||||
):
|
):
|
||||||
((x0, y0), (x1, y1)) = chars[0][1]
|
((x0, y0), (x1, y1)) = chars[0][1]
|
||||||
self.chars = [
|
ltchars = [
|
||||||
LTChar(
|
LTChar(
|
||||||
bbox=Bbox(x0, y0, x1, y1),
|
bbox=Bbox(x0, y0, x1, y1),
|
||||||
char= chars[0][0],
|
char= chars[0][0],
|
||||||
@@ -36,7 +36,7 @@ class LTCharBlock(LTComponent):
|
|||||||
x1 = max([ x0, x1, nx0, nx1 ])
|
x1 = max([ x0, x1, nx0, nx1 ])
|
||||||
y1 = max([ y0, y1, ny0, ny1 ])
|
y1 = max([ y0, y1, ny0, ny1 ])
|
||||||
y0 = min([ y0, y1, ny0, ny1 ])
|
y0 = min([ y0, y1, ny0, ny1 ])
|
||||||
self.chars.append(LTChar(
|
ltchars.append(LTChar(
|
||||||
bbox=Bbox(nx0, ny0, nx1, ny1),
|
bbox=Bbox(nx0, ny0, nx1, ny1),
|
||||||
char= char[0],
|
char= char[0],
|
||||||
textstate=textstate,
|
textstate=textstate,
|
||||||
@@ -44,7 +44,7 @@ class LTCharBlock(LTComponent):
|
|||||||
graphicstate=graphicstate
|
graphicstate=graphicstate
|
||||||
))
|
))
|
||||||
|
|
||||||
LTComponent.__init__(self, Bbox(x0, y0, x1, y1))
|
LTContainer.__init__(self, Bbox(x0, y0, x1, y1), ltchars)
|
||||||
|
|
||||||
self.font = textstate.font
|
self.font = textstate.font
|
||||||
self.textstate = textstate
|
self.textstate = textstate
|
||||||
@@ -60,11 +60,8 @@ class LTCharBlock(LTComponent):
|
|||||||
def __repr__(self):
|
def __repr__(self):
|
||||||
return f"""<{self.__class__.__name__} font="{self.fontname}" text="{self.get_text()}"/>"""
|
return f"""<{self.__class__.__name__} font="{self.fontname}" text="{self.get_text()}"/>"""
|
||||||
|
|
||||||
def __iter__(self):
|
|
||||||
return iter(self.chars)
|
|
||||||
|
|
||||||
def get_text(self):
|
def get_text(self):
|
||||||
return "".join([ c.get_text() for c in self.chars ])
|
return "".join([ c.get_text() for c in self ])
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def fontname(self) -> str:
|
def fontname(self) -> str:
|
||||||
|
|||||||
@@ -90,9 +90,9 @@ class LTComponent(LTItem):
|
|||||||
##
|
##
|
||||||
class LTContainer(LTComponent):
|
class LTContainer(LTComponent):
|
||||||
|
|
||||||
def __init__(self, bbox: Bbox):
|
def __init__(self, bbox: Bbox, objs: List[LTItem] = []):
|
||||||
LTComponent.__init__(self, bbox)
|
LTComponent.__init__(self, bbox)
|
||||||
self._objs: List[LTItem] = []
|
self._objs: List[LTItem] = objs
|
||||||
return
|
return
|
||||||
|
|
||||||
def __iter__(self):
|
def __iter__(self):
|
||||||
|
|||||||
@@ -9,4 +9,5 @@ from .generic import *
|
|||||||
from .lzw import *
|
from .lzw import *
|
||||||
from .matrix import *
|
from .matrix import *
|
||||||
from .text import *
|
from .text import *
|
||||||
from .types import *
|
from .types import *
|
||||||
|
from .log import *
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
import logging
|
||||||
|
|
||||||
|
logging.basicConfig(
|
||||||
|
level=logging.WARNING
|
||||||
|
)
|
||||||
|
|
||||||
|
LOGGERS = {}
|
||||||
|
|
||||||
|
def set_log_level(debug_level: int):
|
||||||
|
logging.basicConfig(level=debug_level)
|
||||||
|
for _, logger in LOGGERS.items():
|
||||||
|
logger.setLevel(debug_level)
|
||||||
|
|
||||||
|
def get_logger(logger_name: str) -> logging.Logger:
|
||||||
|
if logger_name not in LOGGERS.keys():
|
||||||
|
LOGGERS[logger_name] = logging.getLogger(logger_name)
|
||||||
|
return LOGGERS[logger_name]
|
||||||
Regular → Executable
+20
-12
@@ -1,37 +1,45 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
import re
|
import re
|
||||||
import os
|
import os
|
||||||
|
import logging
|
||||||
|
|
||||||
from typing import List
|
from typing import List
|
||||||
from pdfmajor.converters import convert_file
|
from pdfmajor.converters import convert_file
|
||||||
|
from xml.etree import ElementTree
|
||||||
|
from tqdm import tqdm
|
||||||
|
|
||||||
def rel_path(*arg: List[str]) -> str:
|
def rel_path(*arg: List[str]) -> str:
|
||||||
return os.path.join(os.path.dirname(__file__), *arg)
|
return os.path.join(os.path.dirname(__file__), *arg)
|
||||||
|
|
||||||
EXT = 'xml'
|
EXT = 'html'
|
||||||
INPUT_FOLDER = rel_path("./pdf")
|
INPUT_FOLDER = rel_path("./pdf")
|
||||||
OUTPUT_FOLDER = rel_path(f"./output/{EXT}")
|
OUTPUT_FOLDER = rel_path(f"./output/{EXT}")
|
||||||
|
|
||||||
if not os.path.exists(OUTPUT_FOLDER):
|
if not os.path.exists(OUTPUT_FOLDER):
|
||||||
os.makedirs(OUTPUT_FOLDER)
|
os.makedirs(OUTPUT_FOLDER)
|
||||||
|
|
||||||
# files = os.listdir(INPUT_FOLDER)
|
files = os.listdir(INPUT_FOLDER)
|
||||||
# files = ['fonts.pdf']
|
|
||||||
files = ['bar-charts.pdf']
|
|
||||||
# files = ['font-color.pdf']
|
|
||||||
# files = ['tables.pdf']
|
|
||||||
|
|
||||||
TOTAL = len(files)
|
TOTAL = len(files)
|
||||||
for idx, file_name in enumerate(files):
|
waiter = tqdm(enumerate(files), total=TOTAL)
|
||||||
|
for idx, file_name in waiter:
|
||||||
file_name_no_ext = re.sub(r'\.pdf$',"", file_name)
|
file_name_no_ext = re.sub(r'\.pdf$',"", file_name)
|
||||||
|
input_file_path = os.path.join(INPUT_FOLDER, file_name)
|
||||||
|
output_file_path = os.path.join(OUTPUT_FOLDER, file_name_no_ext + "." + EXT)
|
||||||
|
|
||||||
print(f"processing {file_name} {idx}/{TOTAL}")
|
|
||||||
if len(file_name) > len(file_name_no_ext):
|
if len(file_name) > len(file_name_no_ext):
|
||||||
with open(os.path.join(INPUT_FOLDER, file_name), 'rb') as input_file:
|
waiter.set_description(f"Processing {file_name}")
|
||||||
with open(os.path.join(OUTPUT_FOLDER, file_name_no_ext + "." + EXT), 'wb') as output_file:
|
with open(input_file_path, 'rb') as input_file:
|
||||||
|
with open(output_file_path, 'wb') as output_file:
|
||||||
convert_file(
|
convert_file(
|
||||||
input_file=input_file,
|
input_file=input_file,
|
||||||
output_file=output_file,
|
output_file=output_file,
|
||||||
out_type=EXT,
|
out_type=EXT,
|
||||||
debug=True,
|
debug_level=logging.NOTSET,
|
||||||
)
|
)
|
||||||
print(f"processing {idx+1}/{TOTAL}")
|
if EXT == 'xml':
|
||||||
|
waiter.set_description(f" > testing format {file_name}")
|
||||||
|
ElementTree.parse(output_file_path)
|
||||||
|
waiter.refresh()
|
||||||
|
waiter.set_description("Done")
|
||||||
|
waiter.close()
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
import re
|
||||||
|
import os
|
||||||
|
import logging
|
||||||
|
|
||||||
|
from typing import List
|
||||||
|
from pdfmajor.converters import convert_file
|
||||||
|
from xml.etree import ElementTree
|
||||||
|
from tqdm import tqdm
|
||||||
|
|
||||||
|
from pdfmajor.extractor import extract_items_from_pdf
|
||||||
|
from pdfmajor.layouts import LTPage
|
||||||
|
|
||||||
|
def rel_path(*arg: List[str]) -> str:
|
||||||
|
return os.path.join(os.path.dirname(__file__), *arg)
|
||||||
|
|
||||||
|
print(extract_items_from_pdf(rel_path('pdf/chars.pdf')))
|
||||||
|
# for page in extract_items_from_pdf(rel_path('pdf/chars.pdf')):
|
||||||
|
# print('page-start', page)
|
||||||
|
# for item in page:
|
||||||
|
# print(' ', item)
|
||||||
|
# print('page-end', page)
|
||||||
|
|
||||||
Binary file not shown.
Binary file not shown.
@@ -10,7 +10,7 @@ import sys
|
|||||||
with open('README.md') as readme_file:
|
with open('README.md') as readme_file:
|
||||||
SETUP = {
|
SETUP = {
|
||||||
"name": "pdfmajor",
|
"name": "pdfmajor",
|
||||||
"version": "1.1.6",
|
"version": "1.2.0",
|
||||||
"packages": find_packages(include=['pdfmajor*']),
|
"packages": find_packages(include=['pdfmajor*']),
|
||||||
"install_requires": [
|
"install_requires": [
|
||||||
'pycryptodome',
|
'pycryptodome',
|
||||||
|
|||||||
Reference in new issue
Block a user