39 Commits
Author SHA1 Message Date
ari 7e24c64b5b Centralized all exceptions 2019-12-19 22:04:01 -05:00
ari cdcd2eabe2 ensured tests are passing 2019-12-19 21:58:44 -05:00
ari fdd2fbd85a removed old exception files 2019-12-19 20:18:49 -05:00
ari 087012dd81 all errors caught in one location 2019-12-19 20:08:06 -05:00
ari 630b3b2418 removed unused cmd_record 2019-12-12 21:45:22 -05:00
ari 441abc3f55 version bump 2019-12-11 22:18:51 -05:00
ari 152d3aaf55 v1.3.12 2019-12-11 22:16:28 -05:00
ari 36a9371657 generated docs 2019-12-08 14:45:28 -05:00
ari a29d465ea4 added support for ignore bad char flag
- updated unit tests to include
- using the textstate.to_unichr abstraction
2019-12-08 14:44:46 -05:00
ari 841576d9b8 exposed exceptions to top module
updated dependencies
2019-12-07 23:15:55 -05:00
ari bfc0b6ba34 added venv 2019-12-07 23:15:12 -05:00
ari 336f5f1fba propograted exceptions 2019-12-07 22:58:13 -05:00
ari 1b1db13b5b updated deps
- upgraded to Pillow 6.2.0 (passes tests)
- removed unnecessary nose dep
2019-12-07 22:57:57 -05:00
ari 3f561f3488 - removed none-develop dependency on tqdm
- ensured all test pdfs are properly pushed
2019-08-18 18:00:50 -04:00
ari 989de3122e corrected gitignore to not ignore lorem-v5.pdf 2019-08-18 17:59:14 -04:00
ari 20be0aa49e removed dependency on tqdm 2019-08-18 17:55:08 -04:00
ari d37b56a685 Merge branch 'master' of github.com:asosnovsky/pdfmajor 2019-06-10 16:28:23 -04:00
ari 9e923de7ec - added better mypy support
- updated most unit-test to use the unitttest module
- added a single unit-test run file
2019-06-10 16:26:52 -04:00
Ari Sosnovsky 2c9593ac21 Update README.md 2019-04-01 17:37:13 -04:00
Ari Sosnovsky 057cc0cf28 Update README.md 2019-04-01 17:36:36 -04:00
Ari Sosnovsky 82b47f1b19 Update README.md 2019-04-01 17:35:24 -04:00
ari 290de97cb5 - fixed invisible color bug 2019-01-26 23:08:57 -05:00
ari b32aef7fda - fixed the curve ignoring of the color error 2019-01-24 21:38:47 -05:00
ari 14ba20be55 - better repr for curves and colors 2019-01-23 22:12:14 -05:00
ari 18938d91ea - better names for commands 2019-01-23 22:11:58 -05:00
ari eb38854076 updated gitinore 2019-01-23 22:11:46 -05:00
ari 671ae5bc4a running tests before every pip deploy 2019-01-23 22:10:51 -05:00
ari 844408c6e3 - ensured that all tests are working
- added a main test script
2019-01-23 22:06:12 -05:00
ari bab104885f - fixed caching error when running the interpreter multiple times on items with xobjects 2019-01-23 15:56:46 -05:00
ari 98f18c3e32 - corrected caching error
- added lorem ipsum pdf test docs
- introduced a new test for detecting if some unwanted caching is occuring
- muting "Unsupported" warning
2019-01-23 15:56:25 -05:00
ari ac2c462561 - improved html converter 2019-01-22 13:16:46 -05:00
ari 19712f1f0f cleaner html converter 2019-01-22 13:16:17 -05:00
ari ec02a7d38b - yaml converter
- adjusting curve detection
2019-01-16 16:56:14 -05:00
ari 0345ca01ba - slightly more accurate shape detection 2019-01-16 16:55:30 -05:00
ari b36f6bc907 silencing the unknown operator warning 2019-01-16 12:08:15 -05:00
ari f9f9a59793 - added yaml converter 2019-01-16 12:08:03 -05:00
ari b7026793d7 - minor bug fix -- xobj invalid name 2019-01-13 13:13:37 -05:00
ari addd59ed0f updated readme 2019-01-11 15:25:03 -05:00
ari d43b617309 - new doc build 2019-01-11 15:23:15 -05:00
87 changed files with 1406 additions and 647 deletions

No files matched your search

+5 -1
View File
@@ -10,4 +10,8 @@ tests/*.txt
.idea/
.vscode
tests/samples/output
**__pycache**
./samples
**__pycache**\
.mypy_cache
.ipynb_checkpoints
venv
+1 -1
View File
@@ -1 +1 @@
v1.3.2b
v1.3.13
+5 -5
View File
@@ -11,15 +11,15 @@
</tr>
</table>
PDF-Major is a fork of `PDFMiner.six`. It is meant to be a more light-weight implementation that makes fewer assumptions on the data. The ultimate goal of this project is to provide a simpler, faster and more functional library that both casual and low-level developers could build upon.
PDF-Major is a **complete re-write** of `PDFMiner.six`. It is meant to be a more light-weight implementation that makes fewer assumptions on the data. The ultimate goal of this project is to provide a simpler, faster and more functional library that both casual and low-level developers could build upon.
## Documentation
See documentation [here](https://asosnovsky.github.io/pdfmajor/).
See [full API documentation here](https://asosnovsky.github.io/pdfmajor/api/).
## Basic Example
```py
from pdfmajor.interpreter import PageInterpreter
from pdfmajor.interpreter import PDFInterpreter
for page in PDFInterpreter("/path/to/pdf.pdf"):
print("page start", page.page_num)
@@ -29,9 +29,9 @@ for page in PDFInterpreter("/path/to/pdf.pdf"):
```
## Why Another Fork?
## Why a re-write?
`PDFMiner` was designed to run in python 2.7>, `PDFMiner.six` was written to bring cross-version support for python 2 and 3. However, after attempting to build and extend upon `PDFMiner.six` I found it rather difficult to extend and identify additional details regarding the items encoded in the pdf. This library will attempt to expose as much information as possible to end-users, without having them solely rely on to-xml or to-json exports.
`PDFMiner` was designed to run in python 2.7, `PDFMiner.six` was written to bring cross-version support for python 2 and 3. However, after attempting to build and extend upon `PDFMiner.six` I found it rather difficult to extend and identify additional details regarding the items encoded in the pdf. This library will attempt to expose as much information as possible to end-users, without having them solely rely on to-xml or to-json exports.
## Features
+12 -7
View File
@@ -6,13 +6,18 @@ cd ..
./bin/clean.sh
python setup.py sdist bdist_wheel
git add .
git commit
git push
if ./bin/run_tests.sh; then
python setup.py sdist bdist_wheel
git tag $(cat .version)
git push -u origin $(cat .version)
git add .
git commit
git push
twine upload dist/*
git tag $(cat .version)
git push -u origin $(cat .version)
twine upload dist/*
else
echo ">>> Stopped due to test failure!"
fi
+19
View File
@@ -0,0 +1,19 @@
#!/bin/bash
# Get working dir (this will make sure that the script will work the same anywhere you run it)
WK_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
cd $WK_DIR
cd ..
# for filename in ./tests/test*.py; do
# if python $filename ; then
# echo woohoo
# else
# exit 1
# fi
# done
if python ./test.py ; then
echo "PASSED TESTS"
else
exit 1
fi
+1
View File
@@ -37,6 +37,7 @@ This generator-function yields individual pages which contain their respected it
- `password`: [str](#) defaults to None
- `caching`: [bool](#) defaults to True
- `check_extractable`: [bool](#) defaults to True
- `ignore_bad_chars`: [bool](#) defaults to False
- `pagenos`: [List[int]](#) defaults to None
- `debug_level`: [logging.levels](#https://docs.python.org/3/library/logging.html#levels) defaults logging.WARNING
+2 -1
View File
@@ -121,7 +121,7 @@
<h2 id="interpreter">interpreter</h2>
<p>a interpreter of the pdf-standard commands</p>
<h3 id="example">Example</h3>
<pre><code class="py">from pdfmajor.interpreter import PageInterpreter
<pre><code class="py">from pdfmajor.interpreter import PDFInterpreter
for page in PDFInterpreter(&quot;/path/to/pdf.pdf&quot;):
print(&quot;page start&quot;, page.page_num)
@@ -140,6 +140,7 @@ for page in PDFInterpreter(&quot;/path/to/pdf.pdf&quot;):
<li><code>password</code>: <a href="#">str</a> defaults to None </li>
<li><code>caching</code>: <a href="#">bool</a> defaults to True </li>
<li><code>check_extractable</code>: <a href="#">bool</a> defaults to True</li>
<li><code>ignore_bad_chars</code>: <a href="#">bool</a> defaults to False</li>
<li><code>pagenos</code>: <a href="#">List[int]</a> defaults to None</li>
<li><code>debug_level</code>: <a href="#https://docs.python.org/3/library/logging.html#levels">logging.levels</a> defaults logging.WARNING</li>
</ul>
+2 -2
View File
@@ -128,7 +128,7 @@
<h2 id="documentation">Documentation</h2>
<p>See documentation <a href="https://asosnovsky.github.io/pdfmajor/">here</a>.</p>
<h2 id="basic-example">Basic Example</h2>
<pre><code class="py">from pdfmajor.interpreter import PageInterpreter
<pre><code class="py">from pdfmajor.interpreter import PDFInterpreter
for page in PDFInterpreter(&quot;/path/to/pdf.pdf&quot;):
print(&quot;page start&quot;, page.page_num)
@@ -240,5 +240,5 @@ SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.</p>
<!--
MkDocs version : 1.0.4
Build Date UTC : 2018-12-24 06:52:28
Build Date UTC : 2019-12-08 19:44:26
-->
+1 -1
View File
@@ -1 +1 @@
{"config":{"lang":["en"],"prebuild_index":false,"separator":"[\\s\\-]+"},"docs":[{"location":"","text":"PDFMajor Latest Release PDF-Major is a fork of PDFMiner.six . It is meant to be a more light-weight implementation that makes fewer assumptions on the data. The ultimate goal of this project is to provide a simpler, faster and more functional library that both casual and low-level developers could build upon. Documentation See documentation here . Basic Example from pdfmajor.interpreter import PageInterpreter for page in PDFInterpreter(\"/path/to/pdf.pdf\"): print(\"page start\", page.page_num) for item in page: print(\" >\", item) print(\"page end\", page.page_num) Why Another Fork? PDFMiner was designed to run in python 2.7>, PDFMiner.six was written to bring cross-version support for python 2 and 3. However, after attempting to build and extend upon PDFMiner.six I found it rather difficult to extend and identify additional details regarding the items encoded in the pdf. This library will attempt to expose as much information as possible to end-users, without having them solely rely on to-xml or to-json exports. Features Functional extraction method based on generators Parse, analyze, and convert PDF documents. PDF-1.7 specification support. (well, almost) Font-Color extraction Shape fill and stroke color extraction CJK languages and vertical writing scripts support. Various font types (Type1, TrueType, Type3, and CID) support. Basic encryption (RC4) support. Note : We took out the layout-analysis process in this version (there is no more LTTextHorizontal or LTTextVertical). While the mathematics behind the grouping process was sound, the coupling of the layout-analysis process with the parsing and interpretation process produced unfriendly-code. This feature could be brought back by running your own implementation of it on the pdfmajor.interpreter.PageInterpreter class, but at the current time is not supported. How to Install Source Install Python 3.6.4 or newer. clone this repo git clone https://github.com/asosnovsky/pdfmajor * install repo python setup.py install Pypi Install Python 3.6.4 or newer. install repo pip install pdfmajor Terms and Conditions (This is so-called MIT/X License) Copyright (c) 2018-2019 Ariel Sosnovsky Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.","title":"Home"},{"location":"#pdfmajor","text":"Latest Release PDF-Major is a fork of PDFMiner.six . It is meant to be a more light-weight implementation that makes fewer assumptions on the data. The ultimate goal of this project is to provide a simpler, faster and more functional library that both casual and low-level developers could build upon.","title":"PDFMajor"},{"location":"#documentation","text":"See documentation here .","title":"Documentation"},{"location":"#basic-example","text":"from pdfmajor.interpreter import PageInterpreter for page in PDFInterpreter(\"/path/to/pdf.pdf\"): print(\"page start\", page.page_num) for item in page: print(\" >\", item) print(\"page end\", page.page_num)","title":"Basic Example"},{"location":"#why-another-fork","text":"PDFMiner was designed to run in python 2.7>, PDFMiner.six was written to bring cross-version support for python 2 and 3. However, after attempting to build and extend upon PDFMiner.six I found it rather difficult to extend and identify additional details regarding the items encoded in the pdf. This library will attempt to expose as much information as possible to end-users, without having them solely rely on to-xml or to-json exports.","title":"Why Another Fork?"},{"location":"#features","text":"Functional extraction method based on generators Parse, analyze, and convert PDF documents. PDF-1.7 specification support. (well, almost) Font-Color extraction Shape fill and stroke color extraction CJK languages and vertical writing scripts support. Various font types (Type1, TrueType, Type3, and CID) support. Basic encryption (RC4) support. Note : We took out the layout-analysis process in this version (there is no more LTTLine truncated
{"config":{"lang":["en"],"prebuild_index":false,"separator":"[\\s\\-]+"},"docs":[{"location":"","text":"PDFMajor Latest Release PDF-Major is a fork of PDFMiner.six . It is meant to be a more light-weight implementation that makes fewer assumptions on the data. The ultimate goal of this project is to provide a simpler, faster and more functional library that both casual and low-level developers could build upon. Documentation See documentation here . Basic Example from pdfmajor.interpreter import PDFInterpreter for page in PDFInterpreter(\"/path/to/pdf.pdf\"): print(\"page start\", page.page_num) for item in page: print(\" >\", item) print(\"page end\", page.page_num) Why Another Fork? PDFMiner was designed to run in python 2.7>, PDFMiner.six was written to bring cross-version support for python 2 and 3. However, after attempting to build and extend upon PDFMiner.six I found it rather difficult to extend and identify additional details regarding the items encoded in the pdf. This library will attempt to expose as much information as possible to end-users, without having them solely rely on to-xml or to-json exports. Features Functional extraction method based on generators Parse, analyze, and convert PDF documents. PDF-1.7 specification support. (well, almost) Font-Color extraction Shape fill and stroke color extraction CJK languages and vertical writing scripts support. Various font types (Type1, TrueType, Type3, and CID) support. Basic encryption (RC4) support. Note : We took out the layout-analysis process in this version (there is no more LTTextHorizontal or LTTextVertical). While the mathematics behind the grouping process was sound, the coupling of the layout-analysis process with the parsing and interpretation process produced unfriendly-code. This feature could be brought back by running your own implementation of it on the pdfmajor.interpreter.PageInterpreter class, but at the current time is not supported. How to Install Source Install Python 3.6.4 or newer. clone this repo git clone https://github.com/asosnovsky/pdfmajor * install repo python setup.py install Pypi Install Python 3.6.4 or newer. install repo pip install pdfmajor Terms and Conditions (This is so-called MIT/X License) Copyright (c) 2018-2019 Ariel Sosnovsky Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.","title":"Home"},{"location":"#pdfmajor","text":"Latest Release PDF-Major is a fork of PDFMiner.six . It is meant to be a more light-weight implementation that makes fewer assumptions on the data. The ultimate goal of this project is to provide a simpler, faster and more functional library that both casual and low-level developers could build upon.","title":"PDFMajor"},{"location":"#documentation","text":"See documentation here .","title":"Documentation"},{"location":"#basic-example","text":"from pdfmajor.interpreter import PDFInterpreter for page in PDFInterpreter(\"/path/to/pdf.pdf\"): print(\"page start\", page.page_num) for item in page: print(\" >\", item) print(\"page end\", page.page_num)","title":"Basic Example"},{"location":"#why-another-fork","text":"PDFMiner was designed to run in python 2.7>, PDFMiner.six was written to bring cross-version support for python 2 and 3. However, after attempting to build and extend upon PDFMiner.six I found it rather difficult to extend and identify additional details regarding the items encoded in the pdf. This library will attempt to expose as much information as possible to end-users, without having them solely rely on to-xml or to-json exports.","title":"Why Another Fork?"},{"location":"#features","text":"Functional extraction method based on generators Parse, analyze, and convert PDF documents. PDF-1.7 specification support. (well, almost) Font-Color extraction Shape fill and stroke color extraction CJK languages and vertical writing scripts support. Various font types (Type1, TrueType, Type3, and CID) support. Basic encryption (RC4) support. Note : We took out the layout-analysis process in this version (there is no more LTTexLine truncated
+2 -2
View File
@@ -2,12 +2,12 @@
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>None</loc>
<lastmod>2018-12-24</lastmod>
<lastmod>2019-12-08</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
<lastmod>2018-12-24</lastmod>
<lastmod>2019-12-08</lastmod>
<changefreq>daily</changefreq>
</url>
</urlset>
Binary file not shown.
+6 -6
View File
@@ -16,15 +16,15 @@
</tr>
</table>
PDF-Major is a fork of `PDFMiner.six`. It is meant to be a more light-weight implementation that makes fewer assumptions on the data. The ultimate goal of this project is to provide a simpler, faster and more functional library that both casual and low-level developers could build upon.
PDF-Major is a **complete re-write** of `PDFMiner.six`. It is meant to be a more light-weight implementation that makes fewer assumptions on the data. The ultimate goal of this project is to provide a simpler, faster and more functional library that both casual and low-level developers could build upon.
## Documentation
See documentation [here](https://asosnovsky.github.io/pdfmajor/).
See [full API documentation here](https://asosnovsky.github.io/pdfmajor/api/).
## Basic Example
```py
from pdfmajor.interpreter import PageInterpreter
from pdfmajor.interpreter import PDFInterpreter
for page in PDFInterpreter("/path/to/pdf.pdf"):
print("page start", page.page_num)
@@ -34,9 +34,9 @@ for page in PDFInterpreter("/path/to/pdf.pdf"):
```
## Why Another Fork?
## Why a re-write?
`PDFMiner` was designed to run in python 2.7>, `PDFMiner.six` was written to bring cross-version support for python 2 and 3. However, after attempting to build and extend upon `PDFMiner.six` I found it rather difficult to extend and identify additional details regarding the items encoded in the pdf. This library will attempt to expose as much information as possible to end-users, without having them solely rely on to-xml or to-json exports.
`PDFMiner` was designed to run in python 2.7, `PDFMiner.six` was written to bring cross-version support for python 2 and 3. However, after attempting to build and extend upon `PDFMiner.six` I found it rather difficult to extend and identify additional details regarding the items encoded in the pdf. This library will attempt to expose as much information as possible to end-users, without having them solely rely on to-xml or to-json exports.
## Features
@@ -97,7 +97,7 @@ OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
"""
__version__ = (1, 3, 2b)
__version__ = (1, 3, 13)
if __name__ == '__main__':
print(
+31 -9
View File
@@ -2,37 +2,40 @@ import io
import os
import re
from typing import List
from typing import List, Optional
from .html import convert_to_html
from .xml import convert_to_xml
from .json import convert_to_json
from .text import convert_to_text
from .yaml import convert_to_yaml
from ..utils import logging
from ..execptions import ConverterException
def convert_file(
input_file: str,
output_file: str = None,
image_folder_path: str = None,
output_file: Optional[str] = None,
image_folder_path: Optional[str] = None,
codec: str = 'utf-8',
maxpages: int = 0,
password: str = None,
password: Optional[str] = None,
caching: bool = True,
check_extractable: bool = True,
pagenos: List[int] = None,
pagenos: Optional[List[int]] = None,
dont_export_images: bool = False,
ignore_bad_chars: bool = False,
debug_level: int = logging.WARNING,
out_type: str = 'html',
):
if output_file is None:
output_file = os.path.join(
os.path.dirname(input_file.name),
os.path.dirname(input_file),
os.path.basename(
re.sub(r"\.\w+$", "."+out_type, input_file.name)
re.sub(r"\.\w+$", "."+out_type, input_file)
)
)
if re.search(re.escape(out_type) + r'$', output_file ) is None:
raise Exception("Please make sure that the file name and output type match!")
raise ConverterException("Please make sure that the file name and output type match!")
if out_type == 'html':
return convert_to_html(
input_file_path=input_file,
@@ -44,6 +47,7 @@ def convert_file(
password=password,
caching=caching,
check_extractable=check_extractable,
ignore_bad_chars=ignore_bad_chars,
pagenos=pagenos,
debug_level=debug_level,
)
@@ -58,6 +62,7 @@ def convert_file(
caching=caching,
check_extractable=check_extractable,
pagenos=pagenos,
ignore_bad_chars=ignore_bad_chars,
dont_export_images=dont_export_images,
debug_level=debug_level,
)
@@ -72,6 +77,22 @@ def convert_file(
caching=caching,
check_extractable=check_extractable,
pagenos=pagenos,
ignore_bad_chars=ignore_bad_chars,
dont_export_images=dont_export_images,
debug_level=debug_level,
)
elif out_type == 'yaml' or out_type == 'yml':
return convert_to_yaml(
input_file_path=input_file,
output_file_path=output_file,
image_folder_path=image_folder_path,
codec=codec,
maxpages=maxpages,
password=password,
caching=caching,
check_extractable=check_extractable,
pagenos=pagenos,
ignore_bad_chars=ignore_bad_chars,
dont_export_images=dont_export_images,
debug_level=debug_level,
)
@@ -86,7 +107,8 @@ def convert_file(
caching=caching,
check_extractable=check_extractable,
pagenos=pagenos,
ignore_bad_chars=ignore_bad_chars,
dont_export_images=dont_export_images,
debug_level=debug_level,
)
else: raise Exception("Please specify out_type as 'html' or 'xml' or 'json' or 'text'")
else: raise ConverterException("Please specify out_type as 'html' or 'xml' or 'json' or 'text' or 'yaml'")
+135 -102
View File
@@ -1,19 +1,22 @@
from typing import List
from typing import List, Optional
from .writers.html import HTMLMaker
from ..interpreter import PDFInterpreter, PageInterpreter, logging
from ..interpreter import LTImage, LTTextBlock, LTCharBlock, LTChar, LTCurve, LTXObject
from ..interpreter.commands import LTItem
from ..interpreter.commands.state import CurvePath, PDFColor
INF = (1<<31) - 1
def convert_to_html(
input_file_path: str,
output_file_path: str,
image_folder_path: str = None,
image_folder_path: Optional[str] = None,
dont_export_images: bool = False,
codec: str = 'utf-8',
maxpages: int = 0,
password: str = None,
password: Optional[str] = None,
caching: bool = True,
ignore_bad_chars: bool = False,
check_extractable: bool = True,
pagenos: List[int] = None,
debug_level: int = logging.WARNING,
@@ -23,6 +26,7 @@ def convert_to_html(
password=password,
caching=caching,
check_extractable=check_extractable,
ignore_bad_chars=ignore_bad_chars,
pagenos=pagenos,
debug_level=debug_level
)
@@ -43,19 +47,21 @@ def convert_to_html(
render_page(html, page)
def render_page(html: HTMLMaker, ltpage: PageInterpreter):
def render(item: LTItem):
def render(item: LTItem, block = True):
if isinstance(item, LTCurve):
render_curve(html, item)
render_curve(html, ltpage, item)
elif isinstance(item, LTXObject):
with html.elm('div', { "class": 'xobject' }, {
'position': 'absolute',
# 'position': 'absolute',
'left': f'{item.x0}px',
'right': f'{item.x1}px',
'bottom': f'{item.y0}px',
'top': f'{item.y1}px',
'width': f'{item.width}px',
'height': f'{item.height}px',
}):
for child in item:
render(child)
render(child, False)
elif isinstance(item, LTImage):
name = item.name
# if html.imagewriter is not None:
@@ -68,42 +74,42 @@ def render_page(html: HTMLMaker, ltpage: PageInterpreter):
})
elif isinstance(item, LTChar):
with html.elm('span', { 'class': 'char' }, {
'width': item.width,
'min-width': item.width,
'max-width': item.width,
'height': item.height,
'min-height': item.height,
'max-height': item.height,
'width': str(item.width) + "px",
'min-width': str(item.width) + "px",
'max-width': str(item.width) + "px",
'height': str(item.height) + "px",
'min-height': str(item.height) + "px",
'max-height': str(item.height) + "px",
}):
html.write(item.get_text())
elif isinstance(item, LTTextBlock):
for char in item:
render(char)
with html.elm('span', {'class': 'text-block'}):
for char in item:
render(char)
elif isinstance(item, LTCharBlock):
with html.elm('span', {'class': 'char-block'}):
with html.elm('p', { 'class': 'text-block' }, {
'position': 'absolute',
'left': f'{item.x0}',
'bottom': f'{item.y0}',
'font-size': f'{item.size}px',
'font-family': item.fontname,
'font-weight': item.font.font_weight,
'color': get_color(item.color, default="black"),
'width': item.width,
'min-width': item.width,
'max-width': item.width,
'height': item.height,
'min-height': item.height,
'max-height': item.height,
'transform': f'skew({item.font.italic_angle}deg, 0deg)',
'text-align': 'center',
'display': 'flex',
'flex-flow': 'row nowrap',
'justify-content': 'space-between',
# 'padding-left': item.font.leading
}):
for char in item:
render(char)
with html.elm('span', { 'class': 'char-block' }, {
'position': 'absolute',
'left': f'{item.x0}',
'bottom': f'{item.y0}',
'font-size': f'{item.size}px',
'font-family': item.fontname,
'font-weight': item.font.font_weight,
'color': get_color(item.color, default="black"),
'width': str(item.width) + "px",
'min-width': str(item.width) + "px",
'max-width': str(item.width) + "px",
'height': str(item.height) + "px",
'min-height': str(item.height) + "px",
'max-height': str(item.height) + "px",
'transform': f'skew({item.font.italic_angle}deg, 0deg)',
'text-align': 'center',
'display': 'flex',
'flex-flow': 'row nowrap',
'justify-content': 'space-between',
# 'padding-left': item.font.leading
}):
for char in item:
render(char)
with html.elm('div', { "class": 'page', "id": f"page-{ltpage.page_num}" }, {
'width': f'{ltpage.width}px',
@@ -125,69 +131,96 @@ def get_color(col: PDFColor, default: str = ""):
else:
return default
def render_curve(html: HTMLMaker, item: LTCurve):
css = {
'position': 'absolute',
'left': f'{item.x0}px',
'bottom': f'{item.y0}px',
'width': f'{item.width}px',
'height': f'{item.height}px',
}
path = ""
svg_attr = {
'class': 'curve',
'z-index': 2
}
path_css = {
def render_curve(html: HTMLMaker, ltpage: PageInterpreter, item: LTCurve):
path, svg_attr = create_svg_path(item)
stroke_col = get_color(item.stroke)
fill_col = get_color(item.fill)
path_css = {}
if fill_col:
path_css['fill'] = fill_col
if stroke_col:
path_css['stroke'] = stroke_col
}
# Compute ViewBox
p_vals = { "x": [], "y": [] }
for p in item.paths:
for pt in p.points:
p_vals['x'].append(pt.x)
p_vals['y'].append(pt.y)
mx_x = max([*p_vals['x'], 1])
mn_x = min([*p_vals['x'], 0])
mx_y = max([*p_vals['y'], 1])
mn_y = min([*p_vals['y'], 0])
if mx_x == mn_x:
mn_x -= 1
if mx_y == mn_y:
mn_y -= 1
dx = 1/( mx_x - mn_x )
dy = (item.height/item.width)/( mx_y - mn_y )
ROUND_VAL = 5
svg_attr.update({
# "viewBox": f"{min(p_vals['x'])} {min(p_vals['y'])} {max(p_vals['x'])} {max(p_vals['y'])}",
"viewBox": f"{round(mn_x*dx, ROUND_VAL)} {round(mn_y*dy, ROUND_VAL)} {round(mx_x*dx , ROUND_VAL)} {round(mx_y*dy, ROUND_VAL)}",
with html.elm('svg', svg_attr, css={
'border': '1px solid black',
'position': 'absolute',
'left': f'{item.x0}px',
'bottom': f'{ltpage.height - item.y0}px',
'top': f'{ltpage.height - item.y1}px',
'right': f'{item.x1}px',
'width': f'{item.width}px',
'height': f'{item.height}px',
}):
html.singleton('path', {
"d": path,
**path_css
}, {
**path_css
})
# Compute shape
for p in item.paths:
if p.method == CurvePath.METHOD.MOVE_TO:
path += f'M{" ".join( f"{round(dx*pt.x, ROUND_VAL)} {round(dy*pt.y, ROUND_VAL)}" for pt in p.points )} '
if p.method == CurvePath.METHOD.LINE_TO:
path += f'L{" ".join( f"{round(dx*pt.x, ROUND_VAL)} {round(dy*pt.y, ROUND_VAL)}" for pt in p.points )} '
if p.method == CurvePath.METHOD.CLOSE_PATH:
path += 'Z'
stroke_col = get_color(item.stroke)
fill_col = get_color(item.fill)
if fill_col:
path_css['fill'] = fill_col
if stroke_col:
path_css['stroke'] = stroke_col
with html.elm('svg', svg_attr, css):
html.singleton('path', {
"d": path,
**path_css
}, {
**path_css
})
def create_svg_path(item: LTCurve):
path = ""
svg_attr = {
'class': 'curve',
'z-index': 2
}
if item.height > 0:
dy = 1.0/item.height
dx = (item.width/item.height)/max([(item.x1-item.x0),0.01])
svg_attr.update({
"viewBox": "0 0 %s %s" % ( item.width/item.height, 1 )
})
else:
dx = 1.0/(item.x1-item.x0)
dy = 1.0
svg_attr.update({
"viewBox": "0 0 1 1"
})
if item.height > 1:
svg_attr.update({
'height': f'{item.height}px',
})
else:
svg_attr.update({
'height': '1px',
})
if item.width > 1:
svg_attr.update({
'width': f'{item.width}px',
})
else:
svg_attr.update({
'width': '1px',
})
def pX(val: float) -> float:
return ( val-item.x0 )*dx
def pY(val: float) -> float:
return ( item.height- (val-item.y0) )*dy
for p in item.paths:
if p.method == CurvePath.METHOD.MOVE_TO:
path += f'M{" ".join( f"{pX(p.x)} {pY(p.y)}" for p in p.points )} '
elif p.method == CurvePath.METHOD.LINE_TO:
path += f'L{" ".join( f"{pX(p.x)} {pY(p.y)}" for p in p.points )} '
elif p.method == CurvePath.METHOD.CLOSE_PATH:
path += 'Z'
elif p.method == CurvePath.METHOD.CURVE_BOTH_TO:
path += f'C{" ".join( f"{pX(pt.x)} {pY(pt.y)}" for pt in p.points )} '
return path, svg_attr
# def render_textblock(html: HTMLMaker, ltpage: PageInterpreter, item: LTTextBlock):
# (x0, y0, x1, y1) = (INF, INF, -INF, -INF)
# for c_block in item:
# x0 = min(x0, c_block.x0)
# y0 = min(y0, c_block.y0)
# x1 = max(x1, c_block.x1)
# y1 = max(y1, c_block.y1)
# with html.elm('span')
+6 -4
View File
@@ -1,4 +1,4 @@
from typing import List
from typing import List, Optional
from .writers.json import JSONMaker, JSONMakerObject, JSONMakerArray
from ..interpreter import PDFInterpreter, PageInterpreter, logging
@@ -21,14 +21,15 @@ def normalize_color_values(values: list):
def convert_to_json(
input_file_path: str,
output_file_path: str,
image_folder_path: str = None,
image_folder_path: Optional[str] = None,
dont_export_images: bool = False,
codec: str = 'utf-8',
maxpages: int = 0,
password: str = None,
password: str = Optional[None],
caching: bool = True,
check_extractable: bool = True,
pagenos: List[int] = None,
ignore_bad_chars: bool = False,
pagenos: Optional[List[int]] = None,
debug_level: int = logging.WARNING,
):
intepreter = PDFInterpreter(input_file_path,
@@ -36,6 +37,7 @@ def convert_to_json(
password=password,
caching=caching,
check_extractable=check_extractable,
ignore_bad_chars=ignore_bad_chars,
pagenos=pagenos,
debug_level=debug_level
)
+2
View File
@@ -14,6 +14,7 @@ def convert_to_text(
password: str = None,
caching: bool = True,
check_extractable: bool = True,
ignore_bad_chars: bool = False,
pagenos: List[int] = None,
debug_level: int = logging.WARNING,
):
@@ -22,6 +23,7 @@ def convert_to_text(
password=password,
caching=caching,
check_extractable=check_extractable,
ignore_bad_chars=ignore_bad_chars,
pagenos=pagenos,
debug_level=debug_level
)
+10 -9
View File
@@ -1,12 +1,14 @@
from typing import Optional
from io import TextIOWrapper
from contextlib import contextmanager
from pdfmajor.execptions import FileAccessException
class HTMLMaker:
class AccessError(Exception): pass
def __init__(self, file_path: str, codec: str = 'utf-8'):
self.file_path = file_path
self.codec = codec
self.outfile: TextIOWrapper = None
self.outfile: Optional[TextIOWrapper] = None
def __enter__(self):
self.outfile = open(self.file_path, 'wb')
@@ -22,7 +24,7 @@ class HTMLMaker:
text = text.encode(self.codec)
self.outfile.write(text)
else:
raise self.AccessError("Attempting to write to file without an open connection")
raise FileAccessException("Attempting to write to file without an open connection")
def write(self, text: str, lineend: str = '\n', deep_space: str = ' '):
text = deep_space*self.__levels_deep + text + lineend
@@ -42,21 +44,20 @@ class HTMLMaker:
f'{name}: {value}'
for name, value in css.items()
])
attrs = " ".join([
attrs_str = " ".join([
f'{name}="{value}"'
for name, value in attrs.items()
])
if not singleton:
self.write(f"<{tag_name} {attrs}>", lineend=lineend)
self.write(f"<{tag_name} {attrs_str}>", lineend=lineend)
self.__levels_deep += 1
yield
self.__levels_deep -= 1
self.write(f"</{tag_name}>", deep_space=deep_space, lineend='\n' if not nolineend else '')
else:
self.write(f"<{tag_name} {attrs} />", lineend=lineend, deep_space=deep_space)
self.write(f"<{tag_name} {attrs_str} />", lineend=lineend, deep_space=deep_space)
yield
def singleton(self, tag_name: str, attrs: dict = None, css: dict = None):
with self.elm(tag_name, attrs, css, singleton=True):
def singleton(self, tag_name: str, attrs: dict = None, css: dict = None, singleton: bool = True):
with self.elm(tag_name, attrs, css, singleton=singleton):
pass
+9 -8
View File
@@ -1,6 +1,8 @@
from typing import Set
from json import dumps as json_dump
from io import TextIOWrapper
from contextlib import contextmanager
from pdfmajor.execptions import FileAccessException
@contextmanager
def JSONMaker(file_path: str, codec: str = 'utf-8'):
@@ -8,14 +10,12 @@ def JSONMaker(file_path: str, codec: str = 'utf-8'):
with JSONMakerObject(outf, codec=codec) as obj:
yield obj
class JSONMakerObject:
class AccessError(Exception): pass
def __init__(self, outfile: TextIOWrapper, codec: str = 'utf-8', levels_deep: int = 0):
self.outfile: TextIOWrapper = outfile
self.codec = codec
self.levels_deep = levels_deep
self.keys = set()
self.keys: Set[str] = set()
def __enter__(self):
self.write_raw('{')
@@ -32,7 +32,7 @@ class JSONMakerObject:
text = text.encode(self.codec)
self.outfile.write(text)
else:
raise self.AccessError("Attempting to write to file without an open connection")
raise FileAccessException("Attempting to write to file without an open connection")
def write(self, text: str, deep_space: str = ' ', prefix:str = ''):
text = prefix + (deep_space*self.levels_deep) + text
@@ -50,7 +50,8 @@ class JSONMakerObject:
self.write_prop(key, val)
def string(self, key: str, val: str):
self.write_prop(key, f'"{val}"')
val_sant = str(val).replace('"', '\\"')
self.write_prop(key, f'"{val_sant}"')
@contextmanager
def object(self, key: str):
@@ -70,7 +71,6 @@ class JSONMakerObject:
class JSONMakerArray:
class AccessError(Exception): pass
def __init__(self, outfile: TextIOWrapper, codec: str = 'utf-8', levels_deep: int = 0):
self.outfile: TextIOWrapper = outfile
self.codec = codec
@@ -92,7 +92,7 @@ class JSONMakerArray:
text = text.encode(self.codec)
self.outfile.write(text)
else:
raise self.AccessError("Attempting to write to file without an open connection")
raise FileAccessException("Attempting to write to file without an open connection")
def write(self, text: str, deep_space: str = ' ', prefix:str = ''):
text = prefix + (deep_space*self.levels_deep) + text
@@ -109,7 +109,8 @@ class JSONMakerArray:
self.write_prop(val)
def string(self, val: str):
self.write_prop(f'"{val}"')
val_sant = str(val).replace('"', '\\"')
self.write_prop(f'"{val_sant}"')
@contextmanager
def object(self):
+2 -2
View File
@@ -1,8 +1,8 @@
from io import TextIOWrapper
from contextlib import contextmanager
from pdfmajor.execptions import FileAccessException
class XMLMaker:
class AccessError(Exception): pass
def __init__(self, file_path: str, codec: str = 'utf-8'):
self.file_path = file_path
self.codec = codec
@@ -25,7 +25,7 @@ class XMLMaker:
text = text.encode(self.codec)
self.outfile.write(text)
else:
raise self.AccessError("Attempting to write to file without an open connection")
raise FileAccessException("Attempting to write to file without an open connection")
def write(self, text: str, lineend: str = '\n', deep_space: str = ' '):
text = deep_space*self.__levels_deep + text + lineend
+117
View File
@@ -0,0 +1,117 @@
from json import dumps as json_dump
from io import TextIOWrapper
from contextlib import contextmanager
from pdfmajor.execptions import FileAccessException
@contextmanager
def YAMLMaker(file_path: str, codec: str = 'utf-8'):
with open(file_path, 'wb') as outf:
with YAMLMakerObject(outf, codec=codec) as obj:
yield obj
class YAMLMakerWriter:
def __init__(self, outfile: TextIOWrapper, codec: str = 'utf-8', levels_deep: int = -1):
self.outfile: TextIOWrapper = outfile
self.codec = codec
self.levels_deep = levels_deep
def __enter__(self):
if self.levels_deep > 0:
self.__write_raw('')
self.levels_deep += 1
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.levels_deep -= 1
def __write_raw(self, text):
if self.outfile is not None:
if self.codec:
text = text.encode(self.codec)
self.outfile.write(text)
else:
raise FileAccessException("Attempting to write to file without an open connection")
def _write(self, text: str, append_deep: bool = True, deep_space: str = ' ', prefix:str = ''):
if not append_deep:
deep_space = ""
text = prefix + (deep_space*self.levels_deep) + text
self.__write_raw(text)
class YAMLMakerObject(YAMLMakerWriter):
def __init__(self, outfile: TextIOWrapper, codec: str = 'utf-8', levels_deep: int = -1, parent_is_array: bool = False):
YAMLMakerWriter.__init__(self, outfile, codec, levels_deep)
self.keys = set()
self.parent_is_array = parent_is_array
def write(self, key: str, value: str):
if key not in self.keys:
self.keys.add(key)
if len(self.keys) > 1:
self._write(f'{key}: {value}', prefix='\n')
else:
self._write(f'{key}: {value}', prefix='',
append_deep=not self.parent_is_array
)
@contextmanager
def object(self, key: str):
if key not in self.keys:
self.keys.add(key)
if len(self.keys) > 1:
self._write(f'{key}:\n', prefix='\n')
else:
self._write(f'{key}:\n', prefix='', append_deep=False)
with YAMLMakerObject(self.outfile, codec=self.codec, levels_deep=self.levels_deep) as obj:
yield obj
@contextmanager
def array(self, key: str):
if key not in self.keys:
self.keys.add(key)
if len(self.keys) > 1:
self._write(f'{key}:\n', prefix='\n')
else:
self._write(f'{key}:\n', prefix='')
with YAMLMakerArray(self.outfile, codec=self.codec, levels_deep=self.levels_deep) as obj:
yield obj
def place_object(self, key: str, obj: dict):
with self.object(key) as child:
for key, val in obj.items():
child.write(key, val)
def place_array(self, key: str, arr: list):
with self.array(key) as child:
for val in arr:
child.write(val)
class YAMLMakerArray(YAMLMakerWriter):
def __init__(self, outfile: TextIOWrapper, codec: str = 'utf-8', levels_deep: int = -1):
YAMLMakerWriter.__init__(self, outfile, codec, levels_deep)
self.childrencount = 0
def write(self, value: str = ""):
self.childrencount += 1
if self.childrencount > 1:
self._write(f'- {value}', prefix='\n')
else:
self._write(f'- {value}', prefix='')
@contextmanager
def object(self):
self.childrencount += 1
if self.childrencount > 1:
self._write(f'- ', prefix='\n')
else:
self._write(f'- ', prefix='')
with YAMLMakerObject(self.outfile, codec=self.codec, levels_deep=self.levels_deep, parent_is_array=True) as obj:
yield obj
def place_object(self, obj: dict):
with self.object() as child:
for key, val in obj.items():
child.write(key, val)
+2
View File
@@ -45,6 +45,7 @@ def convert_to_xml(
maxpages: int = 0,
password: str = None,
caching: bool = True,
ignore_bad_chars: bool = False,
check_extractable: bool = True,
pagenos: List[int] = None,
debug_level: int = logging.WARNING,
@@ -54,6 +55,7 @@ def convert_to_xml(
password=password,
caching=caching,
check_extractable=check_extractable,
ignore_bad_chars=ignore_bad_chars,
pagenos=pagenos,
debug_level=debug_level
)
+146
View File
@@ -0,0 +1,146 @@
from typing import List
from .writers.yaml import YAMLMaker, YAMLMakerObject, YAMLMakerArray
from ..interpreter import PDFInterpreter, PageInterpreter, logging
from ..interpreter import LTImage, LTTextBlock, LTCharBlock, LTCurve, LTXObject
from ..interpreter.commands import LTItem
from ..interpreter.commands import LTRect, LTLine
from ..interpreter.commands.state import CurvePath, PDFColor
from ..parser.PSStackParser import PSLiteral
from ..utils import enc
def normalize_color_values(values: list):
ret = []
for v in values:
if isinstance(v, PSLiteral):
ret.append( enc(str(v.name)).decode() )
else:
ret.append( enc(str(v)).decode() )
return ret
def convert_to_yaml(
input_file_path: str,
output_file_path: str,
image_folder_path: str = None,
dont_export_images: bool = False,
codec: str = 'utf-8',
maxpages: int = 0,
password: str = None,
caching: bool = True,
check_extractable: bool = True,
ignore_bad_chars: bool = False,
pagenos: List[int] = None,
debug_level: int = logging.WARNING,
):
intepreter = PDFInterpreter(input_file_path,
maxpages=maxpages,
password=password,
caching=caching,
check_extractable=check_extractable,
ignore_bad_chars=ignore_bad_chars,
pagenos=pagenos,
debug_level=debug_level
)
with YAMLMaker(output_file_path, codec=codec) as yaml:
with yaml.array("pages") as arr:
for page in intepreter:
with arr.object() as page_obj:
render_page(page_obj, page)
def render_page(yaml: YAMLMakerObject, ltpage: PageInterpreter):
yaml.write("id", ltpage.page_num)
yaml.write("width", ltpage.width)
yaml.write("height", ltpage.height)
with yaml.array("children") as chil_arr:
for child in ltpage:
with chil_arr.object() as chil_obj:
render(chil_obj, child)
def render(yaml: YAMLMakerObject, item: LTItem):
if isinstance(item, LTCurve):
place_curve(yaml, item)
elif isinstance(item, LTXObject):
yaml.write("type", 'xobject')
yaml.write("x0", item.x0)
yaml.write("x1", item.x1)
yaml.write("y0", item.y0)
yaml.write("y1", item.y1)
with yaml.array("children") as chil_arr:
for child in item:
with chil_arr.object() as chil_obj:
render(chil_obj, child)
elif isinstance(item, LTImage):
# if yaml.imagewriter is not None:
# name = yaml.imagewriter.export_image(item)
yaml.write("type", 'img')
yaml.write("src", enc(item.name, None))
yaml.write("x0", item.x0)
yaml.write("x1", item.x1)
yaml.write("y0", item.y0)
yaml.write("y1", item.y1)
elif isinstance(item, LTTextBlock):
place_text_block(yaml, item)
def place_text_block(yaml: YAMLMakerObject, text_block: LTTextBlock):
yaml.write("type", 'text-block')
yaml.write("x0", text_block.x0)
yaml.write("x1", text_block.x1)
yaml.write("y0", text_block.y0)
yaml.write("y1", text_block.y1)
with yaml.array("char_blocks") as cb_arr:
for cb in text_block:
with cb_arr.object() as cb_obj:
place_char_block(cb_obj, cb)
def place_char_block(yaml: YAMLMakerObject, char_block: LTCharBlock):
yaml.write("size", char_block.size)
yaml.write("x0", char_block.x0)
yaml.write("x1", char_block.x1)
yaml.write("y0", char_block.y0)
yaml.write("y1", char_block.y1)
if char_block.color.color_space is not None:
yaml.place_object("color", {
"type": enc(char_block.color.color_space.name).decode(),
"value": normalize_color_values(char_block.color.values)
})
for key, value in char_block.font.descriptor.items():
if key != "Type" and "FontFile" not in key:
yaml.write(key, value)
yaml.write("text",
"".join([
char.get_text()
for char in char_block
])
)
def place_curve(yaml: YAMLMakerObject, item: LTCurve):
if isinstance(item, LTRect):
yaml.write("type", 'rect')
elif isinstance(item, LTLine):
yaml.write("type", 'line')
else:
yaml.write("type", 'curve')
with yaml.array('paths') as paths_arr:
for path in item.paths:
paths_arr.place_object({
"type": path.method.name,
"points": [
{"x": point.x, "y": point.y}
for point in path.points
]
})
yaml.write("x0", item.x0)
yaml.write("x1", item.x1)
yaml.write("y0", item.y0)
yaml.write("y1", item.y1)
if item.fill.color_space is not None:
yaml.place_object("fill", {
"type": enc(item.fill.color_space.name).decode(),
"value": normalize_color_values(item.fill.values)
})
if item.stroke.color_space is not None:
yaml.place_object("stroke", {
"type": enc(item.stroke.color_space.name).decode(),
"value": normalize_color_values(item.stroke.values)
})
+105
View File
@@ -0,0 +1,105 @@
class PDFMajorException(Exception):
pass
# Interpreter Errors
class InterpreterError(PDFMajorException):
pass
class EmptyDocumentError(InterpreterError):
pass
class CommandProcessorError(InterpreterError):
pass
class InvalidOperation(CommandProcessorError):
pass
class RepeatedCommand(CommandProcessorError):
pass
class FontError(InterpreterError):
pass
class UnicodeNotDefined(FontError):
pass
class MissingFont(FontError):
pass
# Parser Errors
class ParserError(PDFMajorException):
pass
class PDFNoOutlines(ParserError):
pass
class PDFDestinationNotFound(ParserError):
pass
class PDFNoValidXRef(ParserError):
pass
class CMapNotFound(ParserError):
pass
class PSException(ParserError):
pass
class PDFValueError(ParserError):
pass
class PDFObjectNotFound(ParserError):
pass
class PDFException(ParserError):
pass
class PDFTypeError(PDFException):
pass
class PDFSyntaxError(PDFException):
pass
class PDFTextExtractionNotAllowed(PDFException):
pass
class PSEOF(PSException):
pass
class PSSyntaxError(PSException):
pass
class PSTypeError(PSException):
pass
class PSValueError(PSException):
pass
class CCITTExeception(ParserError):
pass
class EOFB(CCITTExeception):
pass
class InvalidData(CCITTExeception):
pass
class ByteSkip(CCITTExeception):
pass
class PDFTextExtractionNotAllowed(ParserError):
pass
# converter error
class ConverterException(PDFMajorException):
pass
class FileAccessException(ConverterException):
pass
+4 -4
View File
@@ -5,9 +5,8 @@ from .commands import PDFStateStack
from .commands import process_command_stream, prep_state
class PageInterpreter:
class PDFInterpreterError(Exception): pass
def __init__(self, page: PDFPage, page_num: int, font_cache: dict = {}):
def __init__(self, page: PDFPage, page_num: int, font_cache: dict = None, ignore_bad_chars = False):
(x0, y0, x1, y1) = page.mediabox
if page.rotate == 90:
ctm = [0, -1, 1, 0, -y0, x1]
@@ -19,7 +18,7 @@ class PageInterpreter:
ctm = [1, 0, 0, 1, -x0, -y0]
# Locals
self.font_cache = font_cache
self.font_cache = font_cache if font_cache is not None else {}
self.page = page
self.page_num = page_num
self.height = y1-y0
@@ -30,7 +29,8 @@ class PageInterpreter:
PDFStateStack(),
ctm=ctm,
resources=page.resources,
font_cache=self.font_cache
font_cache=self.font_cache,
ignore_bad_chars=ignore_bad_chars
)
+8 -6
View File
@@ -1,6 +1,6 @@
from typing import List
from tqdm import tqdm
from pdfmajor.execptions import EmptyDocumentError
from ..parser.PDFPage import PDFPage
from ..utils import set_log_level, get_logger, logging
@@ -15,7 +15,6 @@ from .PageInterpreter import PageInterpreter
log = get_logger(__name__)
class PDFInterpreter:
class AccessError(Exception): pass
def __init__(self,
input_file_path: str,
@@ -25,6 +24,7 @@ class PDFInterpreter:
password: str = None,
pagenos: List[int] = None,
preload: bool = False,
ignore_bad_chars: bool = False,
debug_level: int = logging.WARNING,
):
self.input_file_path = input_file_path
@@ -33,6 +33,7 @@ class PDFInterpreter:
self.check_extractable = check_extractable
self.password = password
self.pagenos = pagenos
self.ignore_bad_chars = ignore_bad_chars
self.debug_level = debug_level
self.__pages = []
if preload:
@@ -53,18 +54,19 @@ class PDFInterpreter:
caching=self.caching,
check_extractable=self.check_extractable
)
for page_num, page in tqdm(enumerate(pages), desc="Reading pages...", disable=self.debug_level > logging.INFO):
self.__pages.append(PageInterpreter(page, page_num, font_cache))
for page_num, page in enumerate(pages):
self.__pages.append(PageInterpreter(page, page_num, font_cache, ignore_bad_chars=self.ignore_bad_chars))
yield self.__pages[-1]
log.info(f"Done Reading {len(self.__pages)} pages.")
set_log_level(logging.WARNING)
if len(self.__pages) == 0:
raise self.AccessError("No pages found in pdf-file")
raise EmptyDocumentError("No pages found in pdf-file")
def __iter__(self):
if len(self.__pages) > 0:
set_log_level(self.debug_level)
for page in tqdm(self.__pages, desc="Reading pages from store...", disable=self.debug_level > logging.INFO):
for page in self.__pages:
yield page
set_log_level(logging.WARNING)
else:
+3 -3
View File
@@ -1,5 +1,6 @@
from typing import List
from pdfmajor.execptions import RepeatedCommand
from pdfmajor.utils import MATRIX_IDENTITY, mult_matrix
from pdfmajor.parser.PSStackParser import literal_name
from pdfmajor.parser.PDFStream import PDFStream, list_value, dict_value
@@ -7,17 +8,16 @@ from pdfmajor.parser.constants import LITERAL_FORM, LITERAL_IMAGE
from .state import PDFStateStack, PDFColorSpace
from .state.Curves import CurveMethod, CurvePath, CurvePoint
class PDFCommands:
commands = {}
class InvalidOperation(Exception): pass
class RepeatedCommand(Exception): pass
@classmethod
def __add(cls, cmd_name: str, cmd: callable):
if not (cmd_name in cls.commands.keys()):
cls.commands[cmd_name] = cmd
return cmd
raise cls.RepeatedCommand(cmd_name)
raise RepeatedCommand(cmd_name)
@classmethod
def add(cls, *cmd_names: List[str]):
+14 -7
View File
@@ -1,5 +1,6 @@
from typing import List
from pdfmajor.execptions import CommandProcessorError
from pdfmajor.parser.PDFContentParser import PDFContentParser, PDFStream
from pdfmajor.parser.PSStackParser import PSKeyword, keyword_name
from pdfmajor.utils import get_logger
@@ -17,11 +18,7 @@ from .utils import init_resources
log = get_logger('process_command_stream')
class CommandProcessorError(Exception): pass
def process_command_stream(streams: List[PDFStream], font_cache: dict = None, state: PDFStateStack = None):
if state is None:
state = PDFStateStack()
if font_cache is None:
font_cache = {}
parser = PDFContentParser(streams)
@@ -29,6 +26,10 @@ def process_command_stream(streams: List[PDFStream], font_cache: dict = None, st
if isinstance(obj, PSKeyword):
name = keyword_name(obj)
method = name.replace('*', '_a').replace('"', '_w').replace("'", '_q')
func = None
args = []
if method == "Do":
pass
if method in PDFCommands.commands.keys():
func = PDFCommands.commands.get(method)
nargs = func.__code__.co_argcount-1
@@ -41,18 +42,23 @@ def process_command_stream(streams: List[PDFStream], font_cache: dict = None, st
else:
func(state)
else:
log.warning('Unknown operator: %r (i.e. %r)' % (name, method))
log.debug('Unknown operator: %r (i.e. %r)' % (name, method))
# history.write(name,
# None if func is None else func.__name__
# , args)
else:
state.argstack.append(obj)
# history.write('arg', None, obj)
for complete_item in state.complete_layout_items:
if isinstance(complete_item, LTXObject):
xobj_state = prep_state(
state=PDFGraphicState(),
state=PDFStateStack(),
ctm=complete_item.t_matrix,
resources=complete_item.resources,
font_cache=font_cache
)
xobj_state.graphics = state.graphics.copy()
for item in process_command_stream(
[complete_item.stream],
font_cache=font_cache,
@@ -62,9 +68,10 @@ def process_command_stream(streams: List[PDFStream], font_cache: dict = None, st
yield complete_item
state.complete_layout_items = []
def prep_state(state: PDFStateStack, ctm: tuple, resources: dict, font_cache: dict) -> PDFStateStack:
def prep_state(state: PDFStateStack, ctm: tuple, resources: dict, font_cache: dict, ignore_bad_chars: bool = False) -> PDFStateStack:
state.t_matrix = ctm
state.resources = resources
state.text.ignore_bad_chars = ignore_bad_chars
# set some global states.
state.graphics.ncolspace = state.graphics.scolspace = None
+104 -103
View File
@@ -1,3 +1,4 @@
from pdfmajor.execptions import InvalidOperation
from pdfmajor.utils import mult_matrix, MATRIX_IDENTITY, apply_matrix_pt, get_logger
from pdfmajor.parser.PSStackParser import literal_name
from pdfmajor.parser.PDFStream import PDFStream, list_value, dict_value
@@ -14,42 +15,42 @@ log = get_logger('commands')
# No Support
# @PDFCommands.add('W','W_a', 'sh', 'ET', 'BX', 'EX' ,'BI', 'ID', 'gs')
# def do_no_support(stack: PDFStateStack) -> PDFStateStack:
# # log.warning("Unsupported command")
# # log.debug("Unsupported command")
# return stack
# marked content operators
@PDFCommands.add('MP')
def do_MP(stack: PDFStateStack, tag) -> PDFStateStack:
log.warning(f"Unsupported command - do_tag {[tag]}")
# self.device.do_tag(tag)
return stack
# @PDFCommands.add('MP')
# def do_MP(stack: PDFStateStack, tag) -> PDFStateStack:
# log.debug(f"Unsupported command - do_tag {[tag]}")
# # self.device.do_tag(tag)
# return stack
@PDFCommands.add('DP')
def do_DP(stack: PDFStateStack, tag, props=None) -> PDFStateStack:
log.warning(f"Unsupported command - do_tag {[tag.name, type(tag), props]}")
# self.device.do_tag(tag)
return stack
# @PDFCommands.add('DP')
# def do_DP(stack: PDFStateStack, tag, props=None) -> PDFStateStack:
# log.debug(f"Unsupported command - do_tag {[tag.name, type(tag), props]}")
# # self.device.do_tag(tag)
# return stack
@PDFCommands.add('BMC')
def do_BMC(stack: PDFStateStack, tag) -> PDFStateStack:
log.warning(f"Unsupported command - begin_tag {[tag.name, type(tag)]}")
# self.device.begin_tag(tag)
return stack
# @PDFCommands.add('BMC')
# def do_BMC(stack: PDFStateStack, tag) -> PDFStateStack:
# log.debug(f"Unsupported command - begin_tag {[tag.name, type(tag)]}")
# # self.device.begin_tag(tag)
# return stack
@PDFCommands.add('BDC')
def do_BDC(stack: PDFStateStack, tag, props=None) -> PDFStateStack:
log.warning(f"Unsupported command - begin_tag {[tag.name, type(tag), props]}")
# self.device.begin_tag(tag)
return stack
# @PDFCommands.add('BDC')
# def do_BDC(stack: PDFStateStack, tag, props=None) -> PDFStateStack:
# log.debug(f"Unsupported command - begin_tag {[tag.name, type(tag), props]}")
# # self.device.begin_tag(tag)
# return stack
@PDFCommands.add('EMC')
def do_EMC(stack: PDFStateStack) -> PDFStateStack:
# self.device.end_tag(tag)
log.warning("Unsupported command - end_tag")
return stack
# @PDFCommands.add('EMC')
# def do_EMC(stack: PDFStateStack) -> PDFStateStack:
# # self.device.end_tag(tag)
# log.debug("Unsupported command - end_tag")
# return stack
@PDFCommands.add('q')
def do_q(stack: PDFStateStack) -> PDFStateStack:
def save_state(stack: PDFStateStack) -> PDFStateStack:
stack.gstack.append([
stack.t_matrix,
stack.text.copy(),
@@ -58,136 +59,136 @@ def do_q(stack: PDFStateStack) -> PDFStateStack:
return stack
@PDFCommands.add('Q')
def do_Q(stack: PDFStateStack) -> PDFStateStack:
def restore_state(stack: PDFStateStack) -> PDFStateStack:
if len(stack.gstack) > 0:
stack.t_matrix, stack.text, stack.graphicstate = stack.gstack.pop()
return stack
# concat-matrix
@PDFCommands.add('cm')
def do_cm(stack: PDFStateStack, a1, b1, c1, d1, e1, f1) -> PDFStateStack:
def concat_transition_matrix(stack: PDFStateStack, a1, b1, c1, d1, e1, f1) -> PDFStateStack:
stack.t_matrix = mult_matrix((a1, b1, c1, d1, e1, f1), stack.t_matrix)
return stack
# setlinewidth
@PDFCommands.add('w')
def do_w(stack: PDFStateStack, linewidth) -> PDFStateStack:
def set_line_width(stack: PDFStateStack, linewidth) -> PDFStateStack:
stack.graphics.linewidth = linewidth
return stack
# setlinecap
@PDFCommands.add('J')
def do_J(stack: PDFStateStack, linecap) -> PDFStateStack:
def set_linecap(stack: PDFStateStack, linecap) -> PDFStateStack:
stack.graphics.linecap = linecap
return stack
# setlinejoin
@PDFCommands.add('j')
def do_j(stack: PDFStateStack, linejoin) -> PDFStateStack:
def set_linejoin(stack: PDFStateStack, linejoin) -> PDFStateStack:
stack.graphics.linejoin = linejoin
return stack
# setmiterlimit
@PDFCommands.add('M')
def do_M(stack: PDFStateStack, miterlimit) -> PDFStateStack:
def set_miterlimit(stack: PDFStateStack, miterlimit) -> PDFStateStack:
stack.graphics.miterlimit = miterlimit
return stack
# setdash
@PDFCommands.add('d')
def do_d(stack: PDFStateStack, dash, phase) -> PDFStateStack:
def set_dash(stack: PDFStateStack, dash, phase) -> PDFStateStack:
stack.graphics.dash = (dash, phase)
return stack
# setintent
@PDFCommands.add('ri')
def do_ri(stack: PDFStateStack, intent) -> PDFStateStack:
def set_intent(stack: PDFStateStack, intent) -> PDFStateStack:
stack.graphics.intent = intent
return stack
# setflatness
@PDFCommands.add('i')
def do_i(stack: PDFStateStack, flatness) -> PDFStateStack:
def set_flatness(stack: PDFStateStack, flatness) -> PDFStateStack:
stack.graphics.flatness = flatness
return stack
# moveto
@PDFCommands.add('m')
def do_m(stack: PDFStateStack, x: float, y: float) -> PDFStateStack:
def curve_moveto(stack: PDFStateStack, x: float, y: float) -> PDFStateStack:
stack.curvestacks.append(CurvePath(
CurvePath.METHOD.MOVE_TO,
CurvePoint(x, y)
CurvePoint(stack.t_matrix, x, y)
))
return stack
# lineto
@PDFCommands.add('l')
def do_l(stack: PDFStateStack, x: float, y: float) -> PDFStateStack:
def curve_lineto(stack: PDFStateStack, x: float, y: float) -> PDFStateStack:
stack.curvestacks.append(CurvePath(
CurvePath.METHOD.LINE_TO,
CurvePoint(x, y)
CurvePoint(stack.t_matrix, x, y)
))
return stack
# curveto
@PDFCommands.add('c')
def do_c(stack: PDFStateStack,
def curve_curveto(stack: PDFStateStack,
x1: float, y1: float,
x2: float, y2: float,
x3: float, y3: float ) -> PDFStateStack:
stack.curvestacks.append(CurvePath(
CurvePath.METHOD.CURVE_BOTH_TO,
CurvePoint(x1, y1),
CurvePoint(x2, y2),
CurvePoint(x3, y3),
CurvePoint(stack.t_matrix, x1, y1),
CurvePoint(stack.t_matrix, x2, y2),
CurvePoint(stack.t_matrix, x3, y3),
))
return stack
# urveto
@PDFCommands.add('v')
def do_v(stack: PDFStateStack,
def curve_urveto(stack: PDFStateStack,
x2: float, y2: float,
x3: float, y3: float) -> PDFStateStack:
stack.curvestacks.append(CurvePath(
CurvePath.METHOD.CURVE_NEXT_TO,
CurvePoint(x2, y2),
CurvePoint(x3, y3),
CurvePoint(stack.t_matrix, x2, y2),
CurvePoint(stack.t_matrix, x3, y3),
))
return stack
# rveto
@PDFCommands.add('y')
def do_y(stack: PDFStateStack,
def curve_rveto(stack: PDFStateStack,
x1: float, y1: float,
x3: float, y3: float) -> PDFStateStack:
stack.curvestacks.append(CurvePath(
CurvePath.METHOD.CURVE_FIRST_TO,
CurvePoint(x1, y1),
CurvePoint(x3, y3)
CurvePoint(stack.t_matrix, x1, y1),
CurvePoint(stack.t_matrix, x3, y3)
))
return stack
# closepath
@PDFCommands.add('h')
def do_h(stack: PDFStateStack) -> PDFStateStack:
def curve_close(stack: PDFStateStack) -> PDFStateStack:
stack.curvestacks.append(CurvePath(CurvePath.METHOD.CLOSE_PATH))
return stack
# rectangle
@PDFCommands.add('re')
def do_re(stack: PDFStateStack, x, y, w, h) -> PDFStateStack:
def curve_rect(stack: PDFStateStack, x, y, w, h) -> PDFStateStack:
stack.curvestacks.extend([
CurvePath(CurvePath.METHOD.MOVE_TO, CurvePoint(x, y) ),
CurvePath(CurvePath.METHOD.LINE_TO, CurvePoint(x+w, y) ),
CurvePath(CurvePath.METHOD.LINE_TO, CurvePoint(x+w, y+h) ),
CurvePath(CurvePath.METHOD.LINE_TO, CurvePoint(x, y+h) ),
CurvePath(CurvePath.METHOD.MOVE_TO, CurvePoint(stack.t_matrix, x, y) ),
CurvePath(CurvePath.METHOD.LINE_TO, CurvePoint(stack.t_matrix, x+w, y) ),
CurvePath(CurvePath.METHOD.LINE_TO, CurvePoint(stack.t_matrix, x+w, y+h) ),
CurvePath(CurvePath.METHOD.LINE_TO, CurvePoint(stack.t_matrix, x, y+h) ),
CurvePath(CurvePath.METHOD.CLOSE_PATH)
])
return stack
# stroke, fill
@PDFCommands.add('S', 'f', 'F', 'B')
def do_complete_path(stack: PDFStateStack) -> PDFStateStack:
def curve_complete_path(stack: PDFStateStack) -> PDFStateStack:
stack.complete_layout_items.append(make_curve(
stack.t_matrix,
stack.graphics.copy(),
@@ -199,7 +200,7 @@ def do_complete_path(stack: PDFStateStack) -> PDFStateStack:
# sroke, fill-even-odd
@PDFCommands.add('f_a', 'B_a')
def do_complete_path_evenodd(stack: PDFStateStack) -> PDFStateStack:
def curve_complete_path_evenodd(stack: PDFStateStack) -> PDFStateStack:
stack.complete_layout_items.append(make_curve(
stack.t_matrix,
stack.graphics.copy(),
@@ -211,7 +212,7 @@ def do_complete_path_evenodd(stack: PDFStateStack) -> PDFStateStack:
# close-and-stroke (do_h, do_S)
@PDFCommands.add('s', 'b')
def do_close_complete(stack: PDFStateStack) -> PDFStateStack:
def curve_close_complete(stack: PDFStateStack) -> PDFStateStack:
stack.curvestacks.append(CurvePath(
CurvePath.METHOD.CLOSE_PATH
))
@@ -226,7 +227,7 @@ def do_close_complete(stack: PDFStateStack) -> PDFStateStack:
# close-fill-and-stroke-even-odd
@PDFCommands.add('b_a')
def do_close_complete_evenodd(stack: PDFStateStack) -> PDFStateStack:
def curve_close_complete_evenodd(stack: PDFStateStack) -> PDFStateStack:
stack.curvestacks.append(CurvePath(
CurvePath.METHOD.CLOSE_PATH
))
@@ -241,32 +242,32 @@ def do_close_complete_evenodd(stack: PDFStateStack) -> PDFStateStack:
# close-only (empty path)
@PDFCommands.add('n')
def do_n(stack: PDFStateStack) -> PDFStateStack:
def curve_close_only(stack: PDFStateStack) -> PDFStateStack:
stack.curvestacks = []
return stack
# setcolorspace-stroking
@PDFCommands.add('CS')
def do_CS(stack: PDFStateStack, name: str) -> PDFStateStack:
def set_stroke_colorspace(stack: PDFStateStack, name: str) -> PDFStateStack:
try:
stack.graphics.scolspace = stack.colorspace_map[literal_name(name)]
return stack
except KeyError:
raise PDFCommands.InvalidOperation('Undefined ColorSpace: %r' % name)
raise InvalidOperation('Undefined ColorSpace: %r' % name)
# setcolorspace-non-strokine
@PDFCommands.add('cs')
def do_cs(stack: PDFStateStack, name: str) -> PDFStateStack:
def set_nonstroke_colorspace(stack: PDFStateStack, name: str) -> PDFStateStack:
try:
stack.graphics.ncolspace = stack.colorspace_map[literal_name(name)]
return stack
except KeyError:
raise PDFCommands.InvalidOperation('Undefined ColorSpace: %r' % name)
raise InvalidOperation('Undefined ColorSpace: %r' % name)
# setgray-stroking
@PDFCommands.add('G')
def do_G(stack: PDFStateStack, gray: float) -> PDFStateStack:
def set_stroke_gray(stack: PDFStateStack, gray: float) -> PDFStateStack:
stack.graphics.set_stroke_color(
stack.colorspace_map['DeviceGray'],
gray
@@ -275,7 +276,7 @@ def do_G(stack: PDFStateStack, gray: float) -> PDFStateStack:
# setgray-non-stroking
@PDFCommands.add('g')
def do_g(stack: PDFStateStack, gray: float) -> PDFStateStack:
def set_nonstroke_gray(stack: PDFStateStack, gray: float) -> PDFStateStack:
stack.graphics.set_nostroke_color(
stack.colorspace_map['DeviceGray'],
gray
@@ -285,7 +286,7 @@ def do_g(stack: PDFStateStack, gray: float) -> PDFStateStack:
# setrgb-stroking
@PDFCommands.add('RG')
def do_RG(stack: PDFStateStack, r, g, b) -> PDFStateStack:
def set_stroke_rgb(stack: PDFStateStack, r, g, b) -> PDFStateStack:
stack.graphics.set_stroke_color(
stack.colorspace_map['DeviceRGB'],
round(r*255),
@@ -296,7 +297,7 @@ def do_RG(stack: PDFStateStack, r, g, b) -> PDFStateStack:
# setrgb-non-stroking
@PDFCommands.add('rg')
def do_rg(stack: PDFStateStack, r, g, b) -> PDFStateStack:
def set_nonstroke_rgb(stack: PDFStateStack, r, g, b) -> PDFStateStack:
stack.graphics.set_nostroke_color(
stack.colorspace_map['DeviceRGB'],
round(r*255),
@@ -307,7 +308,7 @@ def do_rg(stack: PDFStateStack, r, g, b) -> PDFStateStack:
# setcmyk-stroking
@PDFCommands.add('K')
def do_K(stack: PDFStateStack, c, m, y, k) -> PDFStateStack:
def set_stroke_cmyk(stack: PDFStateStack, c, m, y, k) -> PDFStateStack:
stack.graphics.set_stroke_color(
stack.colorspace_map['DeviceCMYK'],
c, m, y, k
@@ -316,7 +317,7 @@ def do_K(stack: PDFStateStack, c, m, y, k) -> PDFStateStack:
# setcmyk-non-stroking
@PDFCommands.add('k')
def do_k(stack: PDFStateStack, c, m, y, k) -> PDFStateStack:
def set_nonstroke_cmyk(stack: PDFStateStack, c, m, y, k) -> PDFStateStack:
stack.graphics.set_stroke_color(
stack.colorspace_map['DeviceCMYK'],
c, m, y, k
@@ -324,9 +325,9 @@ def do_k(stack: PDFStateStack, c, m, y, k) -> PDFStateStack:
return stack
@PDFCommands.add('SCN','SC')
def do_SCN(stack: PDFStateStack) -> PDFStateStack:
def set_stroke_colorspace_custom(stack: PDFStateStack) -> PDFStateStack:
if stack.graphics.scolspace is None:
raise PDFCommands.InvalidOperation('No colorspace specified!')
raise InvalidOperation('No colorspace specified!')
components = stack.pop(stack.graphics.scolspace.ncomponents)
stack.graphics.set_stroke_color(
stack.graphics.scolspace,
@@ -335,9 +336,9 @@ def do_SCN(stack: PDFStateStack) -> PDFStateStack:
return stack
@PDFCommands.add('scn','sc')
def do_scn(stack: PDFStateStack) -> PDFStateStack:
def set_nonstroke_colorspace_custom(stack: PDFStateStack) -> PDFStateStack:
if stack.graphics.ncolspace is None:
raise PDFCommands.InvalidOperation('No colorspace specified!')
raise InvalidOperation('No colorspace specified!')
components = stack.pop(stack.graphics.ncolspace.ncomponents)
stack.graphics.set_nostroke_color(
stack.graphics.ncolspace,
@@ -347,70 +348,70 @@ def do_scn(stack: PDFStateStack) -> PDFStateStack:
# begin-text
@PDFCommands.add('BT')
def do_BT(stack: PDFStateStack) -> PDFStateStack:
def begin_text(stack: PDFStateStack) -> PDFStateStack:
stack.text.matrix = MATRIX_IDENTITY
stack.text.linematrix = [0, 0]
if stack.current_textblock is None:
stack.current_textblock = LTTextBlock()
else:
raise PDFCommands.InvalidOperation("Last TextBlock did not clear")
raise InvalidOperation("Last TextBlock did not clear")
return stack
@PDFCommands.add('ET')
def do_ET(stack: PDFStateStack) -> PDFStateStack:
def end_text(stack: PDFStateStack) -> PDFStateStack:
stack.complete_layout_items.append(stack.current_textblock)
stack.current_textblock = None
return stack
# setcharspace
@PDFCommands.add('Tc')
def do_Tc(stack: PDFStateStack, charspace) -> PDFStateStack:
def set_charspace(stack: PDFStateStack, charspace) -> PDFStateStack:
stack.text.charspace = charspace
return stack
# setwordspace
@PDFCommands.add('Tw')
def do_Tw(stack: PDFStateStack, wordspace) -> PDFStateStack:
def set_wordspace(stack: PDFStateStack, wordspace) -> PDFStateStack:
stack.text.wordspace = wordspace
return stack
# textscale
@PDFCommands.add('Tz')
def do_Tz(stack: PDFStateStack, scaling) -> PDFStateStack:
def set_textscale(stack: PDFStateStack, scaling) -> PDFStateStack:
stack.text.scaling = scaling
return stack
# setleading
@PDFCommands.add('TL')
def do_TL(stack: PDFStateStack, leading) -> PDFStateStack:
def set_text_leading(stack: PDFStateStack, leading) -> PDFStateStack:
stack.text.leading = -leading
return stack
# selectfont
@PDFCommands.add('Tf')
def do_Tf(stack: PDFStateStack, fontid, fontsize) -> PDFStateStack:
def set_font(stack: PDFStateStack, fontid, fontsize) -> PDFStateStack:
stack.text.fontsize = fontsize
try:
stack.text.font = stack.fontmap[literal_name(fontid)]
return stack
except KeyError:
raise PDFCommands.InvalidOperation('Undefined Font id: %r' % fontid)
raise InvalidOperation('Undefined Font id: %r' % fontid)
# setrendering
@PDFCommands.add('Tr')
def do_Tr(stack: PDFStateStack, render) -> PDFStateStack:
def set_text_rendering(stack: PDFStateStack, render) -> PDFStateStack:
stack.text.render = render
return stack
# settextrise
@PDFCommands.add('Ts')
def do_Ts(stack: PDFStateStack, rise) -> PDFStateStack:
def set_textrise(stack: PDFStateStack, rise) -> PDFStateStack:
stack.text.rise = rise
return stack
# text-move
@PDFCommands.add('Td')
def do_Td(stack: PDFStateStack, tx, ty) -> PDFStateStack:
def move_text_x(stack: PDFStateStack, tx, ty) -> PDFStateStack:
(a, b, c, d, e, f) = stack.text.matrix
stack.text.matrix = (a, b, c, d, tx*a+ty*c+e, tx*b+ty*d+f)
stack.text.linematrix = [0, 0]
@@ -418,7 +419,7 @@ def do_Td(stack: PDFStateStack, tx, ty) -> PDFStateStack:
# text-move
@PDFCommands.add('TD')
def do_TD(stack: PDFStateStack, tx, ty) -> PDFStateStack:
def move_text_y(stack: PDFStateStack, tx, ty) -> PDFStateStack:
(a, b, c, d, e, f) = stack.text.matrix
stack.text.matrix = (a, b, c, d, tx*a+ty*c+e, tx*b+ty*d+f)
stack.text.leading = ty
@@ -427,14 +428,14 @@ def do_TD(stack: PDFStateStack, tx, ty) -> PDFStateStack:
# textmatrix
@PDFCommands.add('Tm')
def do_Tm(stack: PDFStateStack, a, b, c, d, e, f) -> PDFStateStack:
def set_text_matrix(stack: PDFStateStack, a, b, c, d, e, f) -> PDFStateStack:
stack.text.matrix = (a, b, c, d, e, f)
stack.text.linematrix = [0, 0]
return stack
# nextline
@PDFCommands.add('T_a')
def do_T_a(stack: PDFStateStack) -> PDFStateStack:
def nextline(stack: PDFStateStack) -> PDFStateStack:
(a, b, c, d, e, f) = stack.text.matrix
stack.text.matrix = (a, b, c, d, stack.text.leading*c+e, stack.text.leading*d+f)
stack.text.linematrix = [0, 0]
@@ -442,11 +443,11 @@ def do_T_a(stack: PDFStateStack) -> PDFStateStack:
# show-pos
@PDFCommands.add('TJ')
def do_TJ(stack: PDFStateStack, seq: bytearray) -> PDFStateStack:
def add_char_blocks(stack: PDFStateStack, seq: bytearray) -> PDFStateStack:
if stack.text.font is None:
raise PDFCommands.InvalidOperation("No Font Specified")
raise InvalidOperation("No Font Specified")
if stack.current_textblock is None:
raise PDFCommands.InvalidOperation("No TextBlock initilized")
raise InvalidOperation("No TextBlock initilized")
stack.current_textblock.add_char_block(
seq,
stack.t_matrix,
@@ -457,37 +458,37 @@ def do_TJ(stack: PDFStateStack, seq: bytearray) -> PDFStateStack:
# show
@PDFCommands.add('Tj')
def do_Tj(stack: PDFStateStack, b: bytes) -> PDFStateStack:
return do_TJ(stack, [b])
def add_char_block(stack: PDFStateStack, b: bytes) -> PDFStateStack:
return add_char_blocks(stack, [b])
# quote
@PDFCommands.add('_q')
def do__q(stack: PDFStateStack, b: bytes) -> PDFStateStack:
return do_TJ(do_T_a(stack), [b])
def add_quoted_char_blocks(stack: PDFStateStack, b: bytes) -> PDFStateStack:
return add_char_blocks(nextline(stack), [b])
# doublequote
@PDFCommands.add('_w')
def do__w(stack: PDFStateStack, wordspace, charspace, b) -> PDFStateStack:
def add_dbquoted_char_blocks(stack: PDFStateStack, wordspace, charspace, b) -> PDFStateStack:
stack.text.wordspace = wordspace
stack.text.charspace = charspace
return do_TJ(stack, [b])
return add_char_blocks(stack, [b])
@PDFCommands.add('EI')
def do_EI(stack: PDFStateStack, obj) -> PDFStateStack:
def insert_image(stack: PDFStateStack, obj) -> PDFStateStack:
if 'W' in obj and 'H' in obj:
stack.complete_layout_items.append(make_image(
obj, stack.ctm
obj, stack.t_matrix
))
return stack
# invoke an XObject
@PDFCommands.add('Do')
def do_Do(stack: PDFStateStack, xobjid) -> PDFStateStack:
def insert_xobject(stack: PDFStateStack, xobjid) -> PDFStateStack:
xobjid = literal_name(xobjid)
try:
xobj = PDFStream.validated_stream(stack.xobjmap[xobjid])
except KeyError:
raise PDFCommands.InvalidOperation('Undefined xobject id: %r' % xobjid)
raise InvalidOperation('Undefined xobject id: %r' % xobjid)
# log.info('Processing xobj: %r', xobj)
subtype = xobj.get('Subtype')
if subtype is LITERAL_FORM and 'BBox' in xobj:
@@ -1,8 +1,14 @@
from enum import Enum
from typing import List
from pdfmajor.utils import apply_matrix_pt
class CurvePoint:
def __init__(self, x: float, y: float):
def __init__(self, ctm: tuple, x: float, y: float):
# old = (x,y)
# (x, y) = apply_matrix_pt(ctm, (x, y))
# if old != (x,y):
# print("CurvePoint", ctm, old)
# print(" >", x, y)
self.x = x
self.y = y
@@ -1,10 +1,10 @@
from typing import List
from typing import Tuple, Optional
from .PDFColorSpace import PDFColorSpace
class PDFColor:
def __init__(self, color_space: PDFColorSpace, *values: List[float]):
self.color_space: PDFColorSpace = color_space
self.values: list = values
def __init__(self, color_space: Optional[PDFColorSpace], *values: float):
self.color_space: Optional[PDFColorSpace] = color_space
self.values: Tuple[float, ...] = values
def copy(self):
return self.__class__(
@@ -13,4 +13,6 @@ class PDFColor:
)
def __repr__(self):
return f"<PDFColor space={self.color_space} values={self.values}/>"
return f"""<PDFColor space="{
None if self.color_space is None else self.color_space.name
}" values={self.values}/>"""
@@ -1,3 +1,4 @@
from typing import Dict
from collections import OrderedDict
class PDFColorSpace(object):
@@ -5,13 +6,12 @@ class PDFColorSpace(object):
def __init__(self, name: str, ncomponents: int):
self.name: str = name
self.ncomponents: int = ncomponents
return
def __repr__(self):
return '<PDFColorSpace: %s, ncomponents=%d>' % (self.name, self.ncomponents)
PREDEFINED_COLORSPACE = OrderedDict()
PREDEFINED_COLORSPACE: Dict[str, PDFColorSpace] = OrderedDict()
for (name, n) in [
('DeviceGray', 1), # default value first
@@ -2,7 +2,6 @@ from .PDFColor import PDFColor, PDFColorSpace
from .PDFColorSpace import PREDEFINED_COLORSPACE
class PDFGraphicState(object):
class InvalidOperation(Exception): pass
def __init__(self):
self.linewidth = 0
@@ -58,7 +58,7 @@ class PDFText(PDFItem):
if needcharspace:
textstate.linematrix[idx] += textstate.charspace
text = textstate.font.to_unichr(cid)
text = textstate.to_unichr(cid)
assert isinstance(text, str), str(type(text))
matrix = translate_matrix(textstate.matrix, textstate.linematrix)
@@ -42,7 +42,7 @@ class TextDecoder:
if needcharspace:
textstate.linematrix[idx] += textstate.charspace
text = textstate.font.to_unichr(cid)
text = textstate.to_unichr(cid)
assert isinstance(text, str), str(type(text))
matrix = translate_matrix(textstate.matrix, textstate.linematrix)
@@ -2,6 +2,7 @@
from abc import abstractmethod
from io import BytesIO
from pdfmajor.execptions import UnicodeNotDefined
from pdfmajor.parser.EncodingDB import EncodingDB
from pdfmajor.parser.PSStackParser import LIT
from pdfmajor.parser.PSStackParser import PSLiteral
@@ -10,11 +11,11 @@ from pdfmajor.parser.PDFStream import resolve1
from pdfmajor.parser.PDFStream import int_value
from pdfmajor.parser.PDFStream import num_value
from pdfmajor.parser.PDFStream import list_value
from pdfmajor.parser.PDFStream import PDFException
from pdfmajor.parser.PDFStream import PDFStream
from pdfmajor.parser.cmapdb import FileUnicodeMap
from pdfmajor.parser.cmapdb import CMapParser
## Literals
##
LITERAL_STANDARD_ENCODING = LIT('StandardEncoding')
@@ -22,8 +23,6 @@ LITERAL_TYPE1C = LIT('Type1C')
class PDFFont(object):
class PDFFontError(PDFException): pass
class PDFUnicodeNotDefined(PDFFontError): pass
def __init__(self, descriptor, widths, default_width=None):
self.descriptor = descriptor
@@ -77,7 +76,7 @@ class PDFFont(object):
except KeyError:
try:
return self.widths[self.to_unichr(cid)] * self.hscale
except (KeyError, self.PDFUnicodeNotDefined):
except (KeyError, UnicodeNotDefined):
return self.default_width * self.hscale
def char_disp(self, cid):
@@ -137,4 +136,4 @@ class PDFSimpleFont(PDFFont):
try:
return self.cid2unicode[cid]
except KeyError:
raise self.PDFUnicodeNotDefined(None, cid)
raise UnicodeNotDefined(None, cid)
@@ -1,13 +1,10 @@
## TrueTypeFont
##
import struct
from pdfmajor.execptions import CMapNotFound
from pdfmajor.parser.cmapdb import FileUnicodeMap
class TrueTypeFont(object):
class CMapNotFound(Exception):
pass
def __init__(self, name, fp):
self.name = name
self.fp = fp
@@ -27,7 +24,7 @@ class TrueTypeFont(object):
def create_unicode_map(self):
if 'cmap' not in self.tables:
raise TrueTypeFont.CMapNotFound
raise CMapNotFound
(base_offset, _) = self.tables['cmap']
fp = self.fp
fp.seek(base_offset)
@@ -1,5 +1,6 @@
from pdfmajor.execptions import PSEOF
from pdfmajor.parser.PSStackParser import PSStackParser
from pdfmajor.parser.PSStackParser import PSEOF, KWD
from pdfmajor.parser.PSStackParser import KWD
from pdfmajor.parser.PSStackParser import PSLiteral
from pdfmajor.parser.PSStackParser import literal_name
from pdfmajor.parser.EncodingDB import name2unicode
@@ -1,3 +1,4 @@
from pdfmajor.execptions import FontError
from pdfmajor.utils import get_logger, settings
from pdfmajor.parser.PSStackParser import literal_name
from pdfmajor.parser.PDFStream import list_value
@@ -16,13 +17,13 @@ def get_font(objid: int, spec: dict, cached_fonts: dict = {}):
log.debug('get_font: create: objid=%r, spec=%r', objid, spec)
if settings.STRICT:
if spec['Type'] is not LITERAL_FONT:
raise PDFFont.PDFFontError('Type is not /Font')
raise FontError('Type is not /Font')
# Create a Font object.
if 'Subtype' in spec:
subtype = literal_name(spec['Subtype'])
else:
if settings.STRICT:
raise PDFFont.PDFFontError('Font Subtype is not specified.')
raise FontError('Font Subtype is not specified.')
subtype = 'Type1'
if subtype in ('Type1', 'MMType1'):
# Type1 Font
@@ -47,7 +48,7 @@ def get_font(objid: int, spec: dict, cached_fonts: dict = {}):
font = get_font(None, subspec, cached_fonts)
else:
if settings.STRICT:
raise PDFFont.PDFFontError('Invalid Font spec: %r' % spec)
raise FontError('Invalid Font spec: %r' % spec)
font = PDFType1Font(spec) # this is so wrong!
if objid:
cached_fonts[objid] = font
@@ -1,5 +1,6 @@
from io import BytesIO
from pdfmajor.execptions import FontError, UnicodeNotDefined, CMapNotFound
from pdfmajor.parser.PSStackParser import literal_name
from pdfmajor.parser.PDFStream import int_value
from pdfmajor.parser.PDFStream import num_value
@@ -24,7 +25,7 @@ class PDFType1Font(PDFSimpleFont):
self.basefont = literal_name(spec['BaseFont'])
except KeyError:
if settings.STRICT:
raise self.PDFFontError('BaseFont is missing')
raise FontError('BaseFont is missing')
self.basefont = 'unknown'
try:
(descriptor, widths) = FontMetricsDB.get_metrics(self.basefont)
@@ -86,7 +87,7 @@ class PDFCIDFont(PDFFont):
self.basefont = literal_name(spec['BaseFont'])
except KeyError:
if strict:
raise self.PDFFontError('BaseFont is missing')
raise FontError('BaseFont is missing')
self.basefont = 'unknown'
self.cidsysteminfo = dict_value(spec.get('CIDSystemInfo', {}))
self.cidcoding = '%s-%s' % (resolve1(self.cidsysteminfo.get('Registry', b'unknown')).decode("latin1"),
@@ -95,19 +96,19 @@ class PDFCIDFont(PDFFont):
name = literal_name(spec['Encoding'])
except KeyError:
if strict:
raise self.PDFFontError('Encoding is unspecified')
raise FontError('Encoding is unspecified')
name = 'unknown'
try:
self.cmap = CMapDB.get_cmap(name)
except CMapDB.CMapNotFound as e:
except CMapNotFound as e:
if strict:
raise self.PDFFontError(e)
raise FontError(e)
self.cmap = CMap()
try:
descriptor = dict_value(spec['FontDescriptor'])
except KeyError:
if strict:
raise self.PDFFontError('FontDescriptor is missing')
raise FontError('FontDescriptor is missing')
descriptor = {}
ttf = None
if 'FontFile2' in descriptor:
@@ -123,12 +124,12 @@ class PDFCIDFont(PDFFont):
if ttf:
try:
self.unicode_map = ttf.create_unicode_map()
except TrueTypeFont.CMapNotFound:
except CMapNotFound:
pass
else:
try:
self.unicode_map = CMapDB.get_unicode_map(self.cidcoding, self.cmap.is_vertical())
except CMapDB.CMapNotFound as e:
except CMapNotFound as e:
pass
self.vertical = self.cmap.is_vertical()
@@ -171,4 +172,4 @@ class PDFCIDFont(PDFFont):
raise KeyError(cid)
return self.unicode_map.get_unichr(cid)
except KeyError:
raise self.PDFUnicodeNotDefined(self.cidcoding, cid)
raise UnicodeNotDefined(self.cidcoding, cid)
@@ -1,5 +1,6 @@
from .PDFFont import PDFFont, get_font
from pdfmajor.execptions import MissingFont, UnicodeNotDefined
from pdfmajor.utils import MATRIX_IDENTITY
from .PDFFont import PDFFont, get_font
## PDFTextState
##
@@ -16,14 +17,27 @@ class PDFTextState(object):
self.rise: int = 0
self.matrix = MATRIX_IDENTITY
self.linematrix = [0, 0]
self.ignore_bad_chars = False
def __repr__(self):
return ('<PDFTextState: font=%r, fontsize=%r, charspace=%r, wordspace=%r, '
' scaling=%r, leading=%r, render=%r, rise=%r, '
' matrix=%r, linematrix=%r>' %
' matrix=%r, linematrix=%r ignore_bad_chars=%r>' %
(self.font, self.fontsize, self.charspace, self.wordspace,
self.scaling, self.leading, self.render, self.rise,
self.matrix, self.linematrix))
self.matrix, self.linematrix, self.ignore_bad_chars))
def to_unichr(self, cid: int) -> str:
if self.font is None:
raise MissingFont
try:
return self.font.to_unichr(cid)
except UnicodeNotDefined as e:
if self.ignore_bad_chars:
return ""
else:
raise e
def copy(self):
obj = self.__class__()
@@ -37,6 +51,7 @@ class PDFTextState(object):
obj.rise = self.rise
obj.matrix = self.matrix
obj.linematrix = self.linematrix
obj.ignore_bad_chars = self.ignore_bad_chars
return obj
def reset_matrices(self):
@@ -24,6 +24,14 @@ class LTCurve(LTComponent):
self.stroke = stroke
self.fill = fill
def __repr__(self):
return ('<%s [%s] fill=%s stroke=%s>' % (
self.__class__.__name__,
str(self.bbox),
self.fill,
self.stroke
))
## LTLine
##
class LTLine(LTCurve):
@@ -13,7 +13,7 @@ class LTComponent(LTItem):
if isinstance(bbox, Bbox):
self.bbox = bbox
else:
raise Exception("Invalid BBox")
raise AssertionError("Expected a BBox")
def __repr__(self):
return ('<%s %s>' %
@@ -90,9 +90,9 @@ class LTComponent(LTItem):
##
class LTContainer(LTComponent):
def __init__(self, bbox: Bbox, objs: List[LTItem] = []):
def __init__(self, bbox: Bbox, objs: List[LTItem] = None):
LTComponent.__init__(self, bbox)
self._objs: List[LTItem] = objs
self._objs: List[LTItem] = [] if objs is None else objs
return
def __iter__(self):
@@ -33,7 +33,7 @@ def __render_string_along(idx: int, seq: bytearray,
if needcharspace:
textstate.linematrix[idx] += textstate.charspace
text = textstate.font.to_unichr(cid)
text = textstate.to_unichr(cid)
assert isinstance(text, str), str(type(text))
matrix = translate_matrix(textstate.matrix, textstate.linematrix)
+3 -3
View File
@@ -4,11 +4,11 @@ import operator
from typing import List
from io import BytesIO
from pdfmajor.execptions import PSEOF, PSTypeError
from ..utils import int2byte, choplist, settings
from .PSStackParser import literal_name
from .PSStackParser import PSStackParser
from .PSStackParser import PSEOF, PSTypeError
from .PSStackParser import KWD
from .PDFStream import PDFStream
@@ -101,9 +101,9 @@ class PDFContentParser(PSStackParser):
obj = PDFStream(d, data)
self.push((pos, obj))
self.push((pos, self.KEYWORD_EI))
except PSTypeError:
except PSTypeError as e:
if settings.STRICT:
raise
raise e
else:
self.push((pos, token))
return
@@ -3,7 +3,6 @@ import hashlib as md5
from Crypto.Cipher import ARC4, AES
from Crypto.Hash import SHA256
from .exceptions import * # pylint: disable=W0614
from ..PDFStream import int_value, str_value, dict_value
from ..PSStackParser import literal_name
from ...utils import int2byte, struct
+7 -5
View File
@@ -1,18 +1,20 @@
import re
from logging import getLogger
from typing import Tuple, Optional, Dict
from pdfmajor.execptions import PSEOF, PDFSyntaxError
from ...utils import settings, choplist, nunpack
from ..PSStackParser import PSEOF
from ..PSStackParser import KWD
from ..PDFParser import PDFStreamParser
from ..PDFStream import PDFStream
from ..PDFStream import dict_value
from .constants import LITERAL_OBJSTM, LITERAL_XREF
from .exceptions import PDFNoValidXRef, PDFSyntaxError
log = getLogger(__name__)
Pos = Tuple[Optional[int], int, int]
## XRefs
##
@@ -27,7 +29,7 @@ class PDFBaseXRef(object):
# Must return
# (strmid, index, genno)
# or (None, pos, genno)
def get_pos(self, objid):
def get_pos(self, objid) -> Pos:
raise KeyError(objid)
@@ -36,7 +38,7 @@ class PDFBaseXRef(object):
class PDFXRef(PDFBaseXRef):
def __init__(self):
self.offsets = {}
self.offsets: Dict[int, Pos] = {}
self.trailer = {}
return
@@ -103,7 +105,7 @@ class PDFXRef(PDFBaseXRef):
try:
return self.offsets[objid]
except KeyError:
raise
raise
## PDFXRefFallback
+13 -5
View File
@@ -4,15 +4,23 @@ from Crypto.Cipher import ARC4, AES
from Crypto.Hash import SHA256
from logging import getLogger
from pdfmajor.execptions import (
ParserError,
PDFNoOutlines,
PDFDestinationNotFound,
PDFNoValidXRef,
PSEOF,
PDFObjectNotFound,
PDFTypeError,
PDFSyntaxError
)
from ...utils import settings, choplist, decode_text
from ..PSStackParser import KWD
from ..PSStackParser import literal_name
from ..PSStackParser import PSEOF
from ..PDFStream import int_value, str_value, dict_value, list_value, decipher_all
from ..PDFStream import PDFObjectNotFound, PDFTypeError
from ..PDFParser import PDFStreamParser, PDFStream, PDFSyntaxError
from ..PDFParser import PDFStreamParser, PDFStream
from .exceptions import *
from .PDFSecurityHandler import PDFStandardSecurityHandler
from .PDFSecurityHandler import PDFStandardSecurityHandlerV4
from .PDFSecurityHandler import PDFStandardSecurityHandlerV5
@@ -181,7 +189,7 @@ class PDFDocument(object):
def getobj(self, objid):
assert objid != 0
if not self.xrefs:
raise PDFException('PDFDocument is not initialized')
raise ParserError('PDFDocument is not initialized')
log.debug('getobj: objid=%r', objid)
if objid in self._cached_objs:
(obj, genno) = self._cached_objs[objid]
-22
View File
@@ -1,22 +0,0 @@
from ..PDFParser import PDFSyntaxError
from ..PDFStream import PDFException
## Exceptions
##
class PDFNoValidXRef(PDFSyntaxError):
pass
class PDFNoOutlines(PDFException):
pass
class PDFDestinationNotFound(PDFException):
pass
class PDFEncryptionError(PDFException):
pass
class PDFPasswordIncorrect(PDFEncryptionError):
pass
class PDFTextExtractionNotAllowed(PDFEncryptionError):
pass
+3 -2
View File
@@ -1,11 +1,12 @@
import logging
from pdfmajor.execptions import PDFTextExtractionNotAllowed, PDFObjectNotFound
from ..utils import settings
from .constants import LIT
from .PDFStream import PDFObjectNotFound
from .PDFStream import resolve1, int_value, list_value, dict_value
from .PDFParser import PDFParser
from .PDFDocument import PDFDocument, PDFTextExtractionNotAllowed
from .PDFDocument import PDFDocument
from ..utils import get_logger
+5 -9
View File
@@ -2,12 +2,13 @@
import logging
from io import BytesIO, TextIOWrapper
from ..utils import settings
from pdfmajor.execptions import (
PSSyntaxError,
PSEOF,
PDFSyntaxError
)
from .PSStackParser import PSStackParser
from .PSStackParser import PSSyntaxError
from .PSStackParser import PSEOF
from .PSStackParser import KWD
from .PDFStream import PDFException
from .PDFStream import PDFStream
from .PDFStream.PDFObjRef import PDFObjRef
from .PDFStream import int_value
@@ -16,11 +17,6 @@ from .PDFStream import dict_value
log = logging.getLogger(__name__)
## Exceptions
##
class PDFSyntaxError(PDFException):
pass
## PDFParser
##
+1 -2
View File
@@ -1,12 +1,11 @@
import zlib
from ..PSStackParser import PSException
from pdfmajor.execptions import PDFValueError, PDFObjectNotFound
from ..PSStackParser import PSObject
from ...utils import lzwdecode, settings
from .constants import * # pylint: disable=W0614
from .types import * # pylint: disable=W0614
from .exceptions import * # pylint: disable=W0614
## PDFObjRef
##
-1
View File
@@ -5,7 +5,6 @@ from ...utils import ascii85decode, asciihexdecode, apply_png_predictor
from .constants import *
from .types import *
from .exceptions import *
from .util import *
from .ccitt import ccittfaxdecode
+9 -17
View File
@@ -12,6 +12,7 @@
import sys
import array
from pdfmajor.execptions import EOFB, InvalidData, ByteSkip
from abc import abstractmethod
def get_bytes(data):
@@ -319,15 +320,6 @@ class CCITTG4Parser(BitParser):
BitParser.add(UNCOMPRESSED, 'T00000', '00000000011')
BitParser.add(UNCOMPRESSED, 'T10000', '00000000010')
class EOFB(Exception):
pass
class InvalidData(Exception):
pass
class ByteSkip(Exception):
pass
def __init__(self, width, bytealign=False):
BitParser.__init__(self)
self.width = width
@@ -340,10 +332,10 @@ class CCITTG4Parser(BitParser):
try:
for m in (128, 64, 32, 16, 8, 4, 2, 1):
self._parse_bit(byte & m)
except self.ByteSkip:
except ByteSkip:
self._accept = self._parse_mode
self._state = self.MODE
except self.EOFB:
except EOFB:
break
return
@@ -363,17 +355,17 @@ class CCITTG4Parser(BitParser):
self._accept = self._parse_uncompressed
return self.UNCOMPRESSED
elif mode == 'e':
raise self.EOFB
raise EOFB
elif isinstance(mode, int):
self._do_vertical(mode)
self._flush_line()
return self.MODE
else:
raise self.InvalidData(mode)
raise InvalidData(mode)
def _parse_horiz1(self, n):
if n is None:
raise self.InvalidData
raise InvalidData
self._n1 += n
if n < 64:
self._n2 = 0
@@ -386,7 +378,7 @@ class CCITTG4Parser(BitParser):
def _parse_horiz2(self, n):
if n is None:
raise self.InvalidData
raise InvalidData
self._n2 += n
if n < 64:
self._color = 1-self._color
@@ -401,7 +393,7 @@ class CCITTG4Parser(BitParser):
def _parse_uncompressed(self, bits):
if not bits:
raise self.InvalidData
raise InvalidData
if bits.startswith('T'):
self._accept = self._parse_mode
self._color = int(bits[1])
@@ -449,7 +441,7 @@ class CCITTG4Parser(BitParser):
self._y += 1
self._reset_line()
if self.bytealign:
raise self.ByteSkip
raise ByteSkip
return
def _do_vertical(self, dx):
-16
View File
@@ -1,16 +0,0 @@
from ..PSStackParser import PSException
class PDFException(PSException):
pass
class PDFTypeError(PDFException):
pass
class PDFValueError(PDFException):
pass
class PDFObjectNotFound(PDFException):
pass
class PDFNotImplementedError(PDFException):
pass
-1
View File
@@ -1,6 +1,5 @@
from .PDFObjRef import PDFObjRef
from ...utils import settings, isnumber, int2byte
from .exceptions import PDFTypeError
from logging import getLogger
@@ -4,9 +4,10 @@ log = logging.getLogger(__name__)
from io import TextIOWrapper
from typing import List, Tuple, Union
from pdfmajor.execptions import PSEOF
from ...utils import int2byte
from .exceptions import PSEOF
from .constants import EOL, NONSPC, KWD, END_LITERAL, LIT, HEX
from .constants import SPC, HEX_PAIR
from .constants import END_NUMBER, END_STRING, END_KEYWORD, END_HEX_STRING
+9 -8
View File
@@ -1,11 +1,12 @@
import logging
from io import TextIOWrapper
from pdfmajor.execptions import PSSyntaxError, PSTypeError, PSEOF
from ...utils import choplist, settings
from .PSBaseParser import PSBaseParser
from .types import *
from .exceptions import *
from .constants import *
log = logging.getLogger(__name__)
@@ -121,9 +122,9 @@ class PSStackParser(PSBaseParser):
# end array
try:
self.push(self.end_type('a'))
except PSTypeError:
except PSTypeError as e:
if settings.STRICT:
raise
raise e
elif token == KEYWORD_DICT_BEGIN:
# begin dictionary
self.start_type(pos, 'd')
@@ -136,9 +137,9 @@ class PSStackParser(PSBaseParser):
# construct a Python dictionary.
d = dict((literal_name(k), v) for (k, v) in choplist(2, objs) if v is not None)
self.push((pos, d))
except PSTypeError:
except PSTypeError as e:
if settings.STRICT:
raise
raise e
elif token == KEYWORD_PROC_BEGIN:
# begin proc
self.start_type(pos, 'p')
@@ -146,16 +147,16 @@ class PSStackParser(PSBaseParser):
# end proc
try:
self.push(self.end_type('p'))
except PSTypeError:
except PSTypeError as e:
if settings.STRICT:
raise
raise e
elif isinstance(token,PSKeyword):
log.debug('do_keyword: pos=%r, token=%r, stack=%r', pos, token, self.curstack)
self.do_keyword(pos, token)
else:
log.error('unknown token: pos=%r, token=%r, stack=%r', pos, token, self.curstack)
self.do_keyword(pos, token)
raise Exception("unknown tokens")
raise PSSyntaxError("unknown tokens")
if self.context:
continue
else:
@@ -1,20 +0,0 @@
## PS Exceptions
##
class PSException(Exception):
pass
class PSEOF(PSException):
pass
class PSSyntaxError(PSException):
pass
class PSTypeError(PSException):
pass
class PSValueError(PSException):
pass
+7 -10
View File
@@ -19,10 +19,13 @@ import pickle as pickle
import struct
import logging
from pdfmajor.execptions import (
CMapNotFound,
PSEOF,
PSSyntaxError,
)
from ...utils import choplist, nunpack, name2unicode
from ..PSStackParser import PSStackParser
from ..PSStackParser import PSSyntaxError
from ..PSStackParser import PSEOF
from ..PSStackParser import PSLiteral
from ..PSStackParser import KWD
from ..PSStackParser import literal_name
@@ -30,9 +33,6 @@ from ..PSStackParser import literal_name
log = logging.getLogger(__name__)
class CMapError(Exception):
pass
## CMapBase
##
@@ -219,9 +219,6 @@ class CMapDB(object):
_cmap_cache = {}
_umap_cache = {}
class CMapNotFound(CMapError):
pass
@classmethod
def _load_data(klass, name):
name = name.replace("\0", "")
@@ -238,7 +235,7 @@ class CMapDB(object):
finally:
gzfile.close()
else:
raise CMapDB.CMapNotFound(name)
raise CMapNotFound(name)
@classmethod
def get_cmap(klass, name):
@@ -325,7 +322,7 @@ class CMapParser(PSStackParser):
self.cmap.use_cmap(CMapDB.get_cmap(literal_name(cmapname)))
except PSSyntaxError:
pass
except CMapDB.CMapNotFound:
except CMapNotFound:
pass
return
+6
View File
@@ -13,6 +13,12 @@ class Bbox:
self.x1 = x1
self.y0 = y0
self.y1 = y1
if (x0 is not None) and (x1 is not None):
self.x0 = min([x0,x1])
self.x1 = max([x0,x1])
if (y0 is not None) and (y1 is not None):
self.y0 = min([y0,y1])
self.y1 = max([y0,y1])
@property
def height(self) -> float:
Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 576 KiB

Binary file not shown.
Binary file not shown.
+10 -3
View File
@@ -10,12 +10,19 @@ import sys
with open('README.md') as readme_file:
SETUP = {
"name": "pdfmajor",
"version": "1.3.2b",
"version": "1.3.13",
"packages": find_packages(include=['pdfmajor*']),
"install_requires": [
'pycryptodome',
'chardet'
'pycryptodome==3.8.2',
'chardet==3.0.4',
'Pillow>=6.2.0'
],
'extras_require': {
'dev': [
'tqdm==4.32.1',
'mkdocs==1.0.4'
]
},
"description": 'PDF parser',
"long_description": readme_file.read(),
'long_description_content_type': 'text/markdown',
+4
View File
@@ -0,0 +1,4 @@
from unittest import TestLoader, TextTestRunner
tests = TestLoader().discover('./tests')
result = TextTestRunner(verbosity=20).run(tests)
-1
View File
@@ -1 +0,0 @@
../pdfmajor
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
+24
View File
@@ -0,0 +1,24 @@
from pathlib import Path
from unittest import TestCase, main
from pdfmajor.interpreter import PDFInterpreter
from pdfmajor.execptions import UnicodeNotDefined
CURRENT_FOLDER = Path("./")
class BadUnicode(TestCase):
def test_ensure_failure(self):
pdf = PDFInterpreter(CURRENT_FOLDER / "tests" / "samples" / "pdf" / "bad-unicode.pdf", ignore_bad_chars=False)
pages = list(pdf)
with self.assertRaises(UnicodeNotDefined):
list(pages[2])
def test_ensure_no_failure(self):
pdf = PDFInterpreter(CURRENT_FOLDER / "tests" / "samples" / "pdf" / "bad-unicode.pdf", ignore_bad_chars=True)
pages = list(pdf)
list(pages[2])
if __name__ == '__main__':
# Run Tests
main()
+38
View File
@@ -0,0 +1,38 @@
import os
from unittest import TestCase, main
# from pdfmajor.extractor import extract_items_from_pdf
from pdfmajor.interpreter import LTCharBlock, PDFInterpreter, logging
from pdfmajor.interpreter import PageInterpreter
from pdfmajor.interpreter.commands import LTContainer
CUR_PATH = os.path.dirname(os.path.dirname(__file__))
INPUT_FOLDER = os.path.join(
CUR_PATH, "tests/samples/pdf"
)
FILES = os.listdir(INPUT_FOLDER)
NUM_LOOPS = 10
def count_elms(file_path: str):
count = 0
for page in PDFInterpreter(file_path, ignore_bad_chars=True):
for item in page:
if isinstance(item, LTContainer):
count += len([*item])
else:
count += 1
return count
class CachingTester(TestCase):
def test_cache(self):
for file_name in FILES:
self.subTest(name = file_name)
file_path = os.path.join(INPUT_FOLDER, file_name)
last_count = count_elms(file_path)
for _ in range(NUM_LOOPS):
new_count = count_elms(file_path)
self.assertEqual( new_count , last_count )
last_count = new_count
if __name__ == '__main__':
# Run Tests
main()
+115
View File
@@ -0,0 +1,115 @@
import os
from typing import List
from unittest import TestCase, main
# from pdfmajor.extractor import extract_items_from_pdf
from pdfmajor.interpreter import LTCharBlock, PDFInterpreter, logging
from pdfmajor.interpreter import PageInterpreter
from pdfmajor.interpreter.commands import LTCurve, PDFColor, LTXObject
from pdfmajor.interpreter.commands import PREDEFINED_COLORSPACE
CUR_PATH = os.path.dirname(os.path.dirname(__file__))
INPUT_FOLDER = os.path.join(
CUR_PATH, "tests/samples/pdf"
)
FILES = os.listdir(INPUT_FOLDER)
NUM_LOOPS = 10
class ColorTest(TestCase):
def run_test(self, file_path: str, curve_defs: List[List[PDFColor]]):
def run_once(item):
if isinstance(item, LTCurve):
t_stroke, t_fill = curve_defs.pop(0)
self.assertEqual( item.stroke.values, t_stroke.values )
self.assertEqual( item.stroke.color_space, t_stroke.color_space )
self.assertEqual( item.fill.values, t_fill.values )
self.assertEqual( item.fill.color_space, t_fill.color_space )
elif isinstance(item, LTXObject):
for c in item:
run_once(c)
for page in PDFInterpreter(file_path):
for item in page:
run_once(item)
def test_lorem_v3(self):
self.run_test(
os.path.join(INPUT_FOLDER, "lorem-v3.pdf"),
[
[
PDFColor(None),
PDFColor(PREDEFINED_COLORSPACE['DeviceRGB'], 255, 255, 255),
],
[
PDFColor(None),
PDFColor(PREDEFINED_COLORSPACE['DeviceRGB'], 114, 159, 207),
],
[
PDFColor(PREDEFINED_COLORSPACE['DeviceRGB'], 255, 255, 0),
PDFColor(PREDEFINED_COLORSPACE['DeviceRGB'], 114, 159, 207),
],
[
PDFColor(PREDEFINED_COLORSPACE['DeviceRGB'], 255, 255, 0),
PDFColor(PREDEFINED_COLORSPACE['DeviceRGB'], 114, 159, 207),
],
[
PDFColor(None),
PDFColor(PREDEFINED_COLORSPACE['DeviceRGB'], 255, 255, 255),
],
[
PDFColor(PREDEFINED_COLORSPACE['DeviceRGB'], 0, 0, 128),
PDFColor(PREDEFINED_COLORSPACE['DeviceRGB'], 255, 0, 0),
]
] +
([[
PDFColor(PREDEFINED_COLORSPACE['DeviceRGB'], 0, 0, 128),
PDFColor(PREDEFINED_COLORSPACE['DeviceRGB'], 255, 255, 255),
]]*24)+
([[
PDFColor(PREDEFINED_COLORSPACE['DeviceRGB'], 0, 0, 128),
PDFColor(PREDEFINED_COLORSPACE['DeviceRGB'], 0, 0, 0),
]]*11)+
([[
PDFColor(PREDEFINED_COLORSPACE['DeviceRGB'], 128, 128, 128),
PDFColor(PREDEFINED_COLORSPACE['DeviceRGB'], 0, 0, 0),
]]*5)+
[[
PDFColor(None),
PDFColor(PREDEFINED_COLORSPACE['DeviceRGB'], 255, 255, 255),
]]
)
def test_lorem_v4(self):
self.run_test(
os.path.join(INPUT_FOLDER, "lorem-v4.pdf"),
[
[
PDFColor(None),
PDFColor(PREDEFINED_COLORSPACE['DeviceRGB'], 255, 255, 255),
],
[
PDFColor(None),
PDFColor(PREDEFINED_COLORSPACE['DeviceRGB'], 64, 64, 64),
]
]
)
def test_lorem_v5(self):
self.run_test(
os.path.join(INPUT_FOLDER, "lorem-v5.pdf"),
[
[
PDFColor(None),
PDFColor(PREDEFINED_COLORSPACE['DeviceRGB'], 255, 255, 255),
],
[
PDFColor(None),
PDFColor(PREDEFINED_COLORSPACE['DeviceRGB'], 128, 128, 128),
]
]
)
if __name__ == '__main__':
# Run Tests
main()
+37 -27
View File
@@ -3,13 +3,14 @@ import re
import os
import logging
from unittest import TestCase, main
from typing import List
from pdfmajor.converters import convert_file
from xml.etree import ElementTree
from json import load as json_load
from tqdm import tqdm
def rel_path(*arg: List[str]) -> str:
def rel_path(*arg: str) -> str:
return os.path.normpath(os.path.join(os.path.dirname(__file__), *arg))
EXTS = ['text', 'json','xml','html']
@@ -22,30 +23,39 @@ if not os.path.exists(OUTPUT_FOLDER):
files = os.listdir(INPUT_FOLDER)
TOTAL = len(files)*len(EXTS)
waiter = tqdm(total=TOTAL)
for idx, file_name in enumerate(files):
file_name_no_ext = re.sub(r'\.pdf$',"", file_name)
input_file_path = os.path.join(INPUT_FOLDER, file_name)
for ext in EXTS:
if not os.path.exists(os.path.join(OUTPUT_FOLDER, ext)):
os.makedirs(os.path.join(OUTPUT_FOLDER, ext))
output_file_path = os.path.join(OUTPUT_FOLDER, ext, file_name_no_ext + "." + ext)
if len(file_name) > len(file_name_no_ext):
waiter.set_description(f"Processing {file_name}")
convert_file(
input_file=input_file_path,
output_file=output_file_path,
out_type=ext,
debug_level=logging.WARN,
)
if ext == 'xml':
waiter.set_description(f" > testing format {file_name}")
ElementTree.parse(output_file_path)
elif ext == 'json':
waiter.set_description(f" > testing format {file_name}")
json_load(open(output_file_path, 'r'))
waiter.refresh()
waiter.update()
waiter.set_description("Done")
waiter.close()
class ConversionTest(TestCase):
def test_conversion(self):
# with tqdm(total=TOTAL) as waiter:
for idx, file_name in enumerate(files):
self.subTest(i=idx);
file_name_no_ext = re.sub(r'\.pdf$',"", file_name)
input_file_path = os.path.join(INPUT_FOLDER, file_name)
for ext in EXTS:
if not os.path.exists(os.path.join(OUTPUT_FOLDER, ext)):
os.makedirs(os.path.join(OUTPUT_FOLDER, ext))
output_file_path = os.path.join(OUTPUT_FOLDER, ext, file_name_no_ext + "." + ext)
if len(file_name) > len(file_name_no_ext):
# waiter.set_description(f"Conversion test -- {file_name}")
convert_file(
input_file=input_file_path,
output_file=output_file_path,
out_type=ext,
ignore_bad_chars=True,
debug_level=logging.WARN,
)
if ext == 'xml':
# waiter.set_description(f" > testing format {file_name}")
ElementTree.parse(output_file_path)
elif ext == 'json':
# waiter.set_description(f" > testing format {file_name}")
with open(output_file_path, 'r') as outf:
json_load(outf)
# waiter.update()
# waiter.set_description("Done")
if __name__ == '__main__':
# Run Tests
main()
+52 -30
View File
@@ -1,43 +1,60 @@
from contextlib import contextmanager
from unittest import TestCase, main
from typing import List, Tuple, Callable
from os import remove
from tempfile import mktemp
from pdfmajor.converters.writers.html import HTMLMaker
from tqdm import tqdm
# Make temp file
FILE_NAME = mktemp()
TestFunction = Callable[[HTMLMaker], str];
TESTS: List[Tuple[TestFunction, str]] = [];
with HTMLMaker(FILE_NAME) as html:
with html.elm("html", nolineend=True):
html.write("test")
assert(open(FILE_NAME, 'r').read() == """<html >
class HTMLTester(TestCase):
@contextmanager
def get_html(self):
with HTMLMaker(FILE_NAME) as html:
yield html
def compare(self, expected: str):
with open(FILE_NAME, 'r') as f:
self.assertEqual(f.read() , expected)
def test_html_basic(self):
with self.get_html() as html:
with html.elm("html", nolineend=True):
html.write("test")
self.compare("""<html >
test
</html>""")
with HTMLMaker(FILE_NAME) as html:
with html.elm("html", nolineend=True):
with html.elm("head"):
html.singleton("meta", {
'http-equiv': "Content-Type",
'content': "text/html"
})
assert(open(FILE_NAME, 'r').read() == """<html >
def test_html_with_attributes(self):
with self.get_html() as html:
with html.elm("html", nolineend=True):
with html.elm("head"):
html.singleton("meta", {
'http-equiv': "Content-Type",
'content': "text/html"
})
self.compare("""<html >
<head >
<meta http-equiv="Content-Type" content="text/html" />
</head>
</html>""")
with HTMLMaker(FILE_NAME) as html:
with html.elm("html", nolineend=True):
with html.elm("head"):
html.singleton("meta", {
'http-equiv': "Content-Type",
'content': "text/html"
})
with html.elm("body"):
with html.elm("h1", css={"color": "red"}):
html.write("Hello World")
assert(open(FILE_NAME, 'r').read() == """<html >
def test_html_with_children(self):
with self.get_html() as html:
with html.elm("html", nolineend=True):
with html.elm("head"):
html.singleton("meta", {
'http-equiv': "Content-Type",
'content': "text/html"
})
with html.elm("body"):
with html.elm("h1", css={"color": "red"}):
html.write("Hello World")
self.compare("""<html >
<head >
<meta http-equiv="Content-Type" content="text/html" />
</head>
@@ -48,13 +65,18 @@ assert(open(FILE_NAME, 'r').read() == """<html >
</body>
</html>""")
with HTMLMaker(FILE_NAME) as html:
with html.elm('style'):
html.write('''.page { position: relative; overflow: hidden;border: 1px black solid;}''')
assert(open(FILE_NAME, 'r').read() == """<style >
def test_html_with_css(self):
with self.get_html() as html:
with html.elm('style'):
html.write('''.page { position: relative; overflow: hidden;border: 1px black solid;}''')
self.compare("""<style >
.page { position: relative; overflow: hidden;border: 1px black solid;}
</style>
""")
# Clean up
remove(FILE_NAME)
if __name__ == '__main__':
# Run Tests
main()
# Clean up
remove(FILE_NAME)
+21 -13
View File
@@ -1,4 +1,5 @@
import os
from unittest import TestCase, main
from pdfmajor.interpreter import LTImage, logging
from pdfmajor.interpreter import PageInterpreter, PDFInterpreter
@@ -18,16 +19,23 @@ OUTPUT_FOLDER = os.path.join(
if not os.path.exists(OUTPUT_FOLDER):
os.makedirs(OUTPUT_FOLDER)
for file_name in FILES:
for page in PDFInterpreter(os.path.join(
CUR_PATH,
f"samples/pdf/{file_name}.pdf"
), debug_level=logging.ERROR):
assert isinstance(page, PageInterpreter)
for item_idx, item in enumerate(page):
assert isinstance(item, LTItem)
if isinstance(item, LTImage):
print(item.save_image(
OUTPUT_FOLDER,
file_name=f'{file_name}_p{page.page_num}i{item_idx}'
))
class ImageTester(TestCase):
def test_image_extract(self):
for file_name in FILES:
self.subTest(file_name=file_name)
for page in PDFInterpreter(os.path.join(
CUR_PATH,
f"tests/samples/pdf/{file_name}.pdf"
), debug_level=logging.ERROR):
self.assertTrue( isinstance(page, PageInterpreter) )
for item_idx, item in enumerate(page):
self.assertTrue( isinstance(item, LTItem) )
if isinstance(item, LTImage):
item.save_image(
OUTPUT_FOLDER,
file_name=f'{file_name}_p{page.page_num}i{item_idx}'
)
if __name__ == '__main__':
# Run Tests
main()
+20 -12
View File
@@ -1,21 +1,29 @@
import os
# from pdfmajor.extractor import extract_items_from_pdf
from unittest import TestCase, main
from pdfmajor.interpreter import LTCharBlock, PDFInterpreter, logging
from pdfmajor.interpreter import PageInterpreter
from pdfmajor.interpreter.commands import LTItem
from pdfmajor.interpreter.commands import LTCharBlock, LTChar
CUR_PATH = os.path.dirname(os.path.dirname(__file__))
FILE_NAME = os.path.join(
CUR_PATH,
"samples/pdf/tables.pdf"
INPUT_FOLDER = os.path.join(
CUR_PATH, "tests/samples/pdf"
)
FILES = os.listdir(INPUT_FOLDER)
for page in PDFInterpreter(FILE_NAME, debug_level=logging.INFO):
assert isinstance(page, PageInterpreter)
for item in page:
assert isinstance(item, LTItem)
if isinstance(item, LTCharBlock):
for char in item:
assert(char, LTChar)
class TestInterpreter(TestCase):
def test_pdfintp(self):
for file_name in FILES:
self.subTest(file_name=file_name)
file_path = os.path.join(INPUT_FOLDER, file_name)
for page in PDFInterpreter(file_path, debug_level=logging.ERROR, ignore_bad_chars=True):
self.assertTrue( isinstance(page, PageInterpreter) )
for item in page:
self.assertTrue( isinstance(item, LTItem) )
if isinstance(item, LTCharBlock):
for char in item:
self.assertTrue( isinstance(char, LTChar) )
if __name__ == '__main__':
# Run Tests
main()
-1
View File
@@ -80,7 +80,6 @@ with JSONMaker(FILE_NAME) as obj:
item.number("x1", 4.2)
item.number("y0", 231.12)
item.number("y1", 331.12)
print(open(FILE_NAME, 'r').read())
assert(open(FILE_NAME, 'r').read() == """{
"pages": [
{
+44 -49
View File
@@ -2,17 +2,12 @@
# -*- coding: utf-8 -*-
from nose.tools import assert_equal, assert_true, assert_false
from nose import SkipTest
import nose
import logging
import unittest
from pdfmajor.parser.PDFStream.ccitt import CCITTG4Parser
## Test cases
##
class TestCCITTG4Parser():
class TestCCITTG4Parser(unittest.TestCase):
def get_parser(self, bits):
parser = CCITTG4Parser(len(bits))
@@ -23,98 +18,98 @@ class TestCCITTG4Parser():
def test_b1(self):
parser = self.get_parser('00000')
parser._do_vertical(0)
assert_equal(parser._curpos, 0)
self.assertEqual(parser._curpos, 0)
return
def test_b2(self):
parser = self.get_parser('10000')
parser._do_vertical(-1)
assert_equal(parser._curpos, 0)
self.assertEqual(parser._curpos, 0)
return
def test_b3(self):
parser = self.get_parser('000111')
parser._do_pass()
assert_equal(parser._curpos, 3)
assert_equal(parser._get_bits(), '111')
self.assertEqual(parser._curpos, 3)
self.assertEqual(parser._get_bits(), '111')
return
def test_b4(self):
parser = self.get_parser('00000')
parser._do_vertical(+2)
assert_equal(parser._curpos, 2)
assert_equal(parser._get_bits(), '11')
self.assertEqual(parser._curpos, 2)
self.assertEqual(parser._get_bits(), '11')
return
def test_b5(self):
parser = self.get_parser('11111111100')
parser._do_horizontal(0, 3)
assert_equal(parser._curpos, 3)
self.assertEqual(parser._curpos, 3)
parser._do_vertical(1)
assert_equal(parser._curpos, 10)
assert_equal(parser._get_bits(), '0001111111')
self.assertEqual(parser._curpos, 10)
self.assertEqual(parser._get_bits(), '0001111111')
return
def test_e1(self):
parser = self.get_parser('10000')
parser._do_vertical(0)
assert_equal(parser._curpos, 1)
self.assertEqual(parser._curpos, 1)
parser._do_vertical(0)
assert_equal(parser._curpos, 5)
assert_equal(parser._get_bits(), '10000')
self.assertEqual(parser._curpos, 5)
self.assertEqual(parser._get_bits(), '10000')
return
def test_e2(self):
parser = self.get_parser('10011')
parser._do_vertical(0)
assert_equal(parser._curpos, 1)
self.assertEqual(parser._curpos, 1)
parser._do_vertical(2)
assert_equal(parser._curpos, 5)
assert_equal(parser._get_bits(), '10000')
self.assertEqual(parser._curpos, 5)
self.assertEqual(parser._get_bits(), '10000')
return
def test_e3(self):
parser = self.get_parser('011111')
parser._color = 0
parser._do_vertical(0)
assert_equal(parser._color, 1)
assert_equal(parser._curpos, 1)
self.assertEqual(parser._color, 1)
self.assertEqual(parser._curpos, 1)
parser._do_vertical(-2)
assert_equal(parser._color, 0)
assert_equal(parser._curpos, 4)
self.assertEqual(parser._color, 0)
self.assertEqual(parser._curpos, 4)
parser._do_vertical(0)
assert_equal(parser._curpos, 6)
assert_equal(parser._get_bits(), '011100')
self.assertEqual(parser._curpos, 6)
self.assertEqual(parser._get_bits(), '011100')
return
def test_e4(self):
parser = self.get_parser('10000')
parser._do_vertical(0)
assert_equal(parser._curpos, 1)
self.assertEqual(parser._curpos, 1)
parser._do_vertical(-2)
assert_equal(parser._curpos, 3)
self.assertEqual(parser._curpos, 3)
parser._do_vertical(0)
assert_equal(parser._curpos, 5)
assert_equal(parser._get_bits(), '10011')
self.assertEqual(parser._curpos, 5)
self.assertEqual(parser._get_bits(), '10011')
return
def test_e5(self):
parser = self.get_parser('011000')
parser._color = 0
parser._do_vertical(0)
assert_equal(parser._curpos, 1)
self.assertEqual(parser._curpos, 1)
parser._do_vertical(3)
assert_equal(parser._curpos, 6)
assert_equal(parser._get_bits(), '011111')
self.assertEqual(parser._curpos, 6)
self.assertEqual(parser._get_bits(), '011111')
return
def test_e6(self):
parser = self.get_parser('11001')
parser._do_pass()
assert_equal(parser._curpos, 4)
self.assertEqual(parser._curpos, 4)
parser._do_vertical(0)
assert_equal(parser._curpos, 5)
assert_equal(parser._get_bits(), '11111')
self.assertEqual(parser._curpos, 5)
self.assertEqual(parser._get_bits(), '11111')
return
def test_e7(self):
@@ -122,8 +117,8 @@ class TestCCITTG4Parser():
parser._curpos = 2
parser._color = 1
parser._do_horizontal(2, 6)
assert_equal(parser._curpos, 10)
assert_equal(parser._get_bits(), '1111000000')
self.assertEqual(parser._curpos, 10)
self.assertEqual(parser._get_bits(), '1111000000')
return
def test_e8(self):
@@ -131,19 +126,19 @@ class TestCCITTG4Parser():
parser._curpos = 1
parser._color = 0
parser._do_vertical(0)
assert_equal(parser._curpos, 2)
self.assertEqual(parser._curpos, 2)
parser._do_horizontal(7, 0)
assert_equal(parser._curpos, 9)
assert_equal(parser._get_bits(), '101111111')
self.assertEqual(parser._curpos, 9)
self.assertEqual(parser._get_bits(), '101111111')
return
def test_m1(self):
parser = self.get_parser('10101')
parser._do_pass()
assert_equal(parser._curpos, 2)
self.assertEqual(parser._curpos, 2)
parser._do_pass()
assert_equal(parser._curpos, 4)
assert_equal(parser._get_bits(), '1111')
self.assertEqual(parser._curpos, 4)
self.assertEqual(parser._get_bits(), '1111')
return
def test_m2(self):
@@ -152,7 +147,7 @@ class TestCCITTG4Parser():
parser._do_vertical(-1)
parser._do_vertical(1)
parser._do_horizontal(1, 1)
assert_equal(parser._get_bits(), '011101')
self.assertEqual(parser._get_bits(), '011101')
return
def test_m3(self):
@@ -161,8 +156,8 @@ class TestCCITTG4Parser():
parser._do_pass()
parser._do_vertical(1)
parser._do_vertical(1)
assert_equal(parser._get_bits(), '00000001')
self.assertEqual(parser._get_bits(), '00000001')
return
if __name__ == '__main__':
nose.runmodule()
unittest.main()
+12 -14
View File
@@ -1,9 +1,7 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from nose.tools import assert_equal
from nose import SkipTest
import nose
from unittest import TestCase, main
#test of various compression/encoding modules (previously in doctests):
from pdfmajor.utils.ascii85 import ascii85decode, asciihexdecode
@@ -14,24 +12,24 @@ import binascii
def hex(b): return binascii.hexlify(b) #encode('hex')
def dehex(b): return binascii.unhexlify(b) #decode('hex')
class TestAscii85():
class TestAscii85(TestCase):
def test_ascii85decode(self):
#The sample string is taken from: http://en.wikipedia.org/w/index.php?title=Ascii85
assert_equal(ascii85decode(b'9jqo^BlbD-BleB1DJ+*+F(f,q'),b'Man is distinguished')
assert_equal(ascii85decode(b'E,9)oF*2M7/c~>'),b'pleasure.')
self.assertEqual(ascii85decode(b'9jqo^BlbD-BleB1DJ+*+F(f,q'),b'Man is distinguished')
self.assertEqual(ascii85decode(b'E,9)oF*2M7/c~>'),b'pleasure.')
def test_asciihexdecode(self):
assert_equal(asciihexdecode(b'61 62 2e6364 65'),b'ab.cde')
assert_equal(asciihexdecode(b'61 62 2e6364 657>'),b'ab.cdep')
assert_equal(asciihexdecode(b'7>'),b'p')
self.assertEqual(asciihexdecode(b'61 62 2e6364 65'),b'ab.cde')
self.assertEqual(asciihexdecode(b'61 62 2e6364 657>'),b'ab.cdep')
self.assertEqual(asciihexdecode(b'7>'),b'p')
class TestLzw():
class TestLzw(TestCase):
def test_lzwdecode(self):
assert_equal(lzwdecode(b'\x80\x0b\x60\x50\x22\x0c\x0c\x85\x01'),b'\x2d\x2d\x2d\x2d\x2d\x41\x2d\x2d\x2d\x42')
self.assertEqual(lzwdecode(b'\x80\x0b\x60\x50\x22\x0c\x0c\x85\x01'),b'\x2d\x2d\x2d\x2d\x2d\x41\x2d\x2d\x2d\x42')
class TestRunlength():
class TestRunlength(TestCase):
def test_rldecode(self):
assert_equal(rldecode(b'\x05123456\xfa7\x04abcde\x80junk'),b'1234567777777abcde')
self.assertEqual(rldecode(b'\x05123456\xfa7\x04abcde\x80junk'),b'1234567777777abcde')
if __name__ == '__main__':
nose.runmodule()
main()
+35 -45
View File
@@ -2,19 +2,15 @@
# -*- coding: utf-8 -*-
from nose.tools import assert_equal, assert_true, assert_false
from nose import SkipTest
import nose
import logging
from pdfmajor.parser.PSStackParser import KWD, LIT, PSBaseParser, PSEOF, PSStackParser
from unittest import TestCase, main
from io import BytesIO
from pdfmajor.parser.PSStackParser import KWD, LIT, PSBaseParser, PSStackParser
from pdfmajor.execptions import PSEOF
## Simplistic Test cases
##
class TestPSBaseParser:
TESTDATA = br'''%!PS
TESTDATA = br'''%!PS
begin end
" @ #
/a/BCD /Some_Name /foo#5f#xbaa
@@ -33,36 +29,32 @@ func/a/b{(c)do*}def
[ 1 (z) ! ]
<< /foo (bar) >>
'''
TOKENS = [
(5, KWD(b'begin')), (11, KWD(b'end')), (16, KWD(b'"')), (19, KWD(b'@')),
(21, KWD(b'#')), (23, LIT('a')), (25, LIT('BCD')), (30, LIT('Some_Name')),
(41, LIT('foo_xbaa')), (54, 0), (56, 1), (59, -2), (62, 0.5),
(65, 1.234), (71, b'abc'), (77, b''), (80, b'abc ( def ) ghi'),
(98, b'def \x00 4ghi'), (118, b'bach\\slask'), (132, b'foo\nbaa'),
(143, b'this % is not a comment.'), (170, b'foo\nbaa'), (180, b'foobaa'),
(191, b''), (194, b' '), (199, b'@@ '), (211, b'\xab\xcd\x00\x124\x05'),
(226, KWD(b'func')), (230, LIT('a')), (232, LIT('b')),
(234, KWD(b'{')), (235, b'c'), (238, KWD(b'do*')), (241, KWD(b'}')),
(242, KWD(b'def')), (246, KWD(b'[')), (248, 1), (250, b'z'), (254, KWD(b'!')),
(256, KWD(b']')), (258, KWD(b'<<')), (261, LIT('foo')), (266, b'bar'),
(272, KWD(b'>>'))
]
OBJS = [
(23, LIT('a')), (25, LIT('BCD')), (30, LIT('Some_Name')),
(41, LIT('foo_xbaa')), (54, 0), (56, 1), (59, -2), (62, 0.5),
(65, 1.234), (71, b'abc'), (77, b''), (80, b'abc ( def ) ghi'),
(98, b'def \x00 4ghi'), (118, b'bach\\slask'), (132, b'foo\nbaa'),
(143, b'this % is not a comment.'), (170, b'foo\nbaa'), (180, b'foobaa'),
(191, b''), (194, b' '), (199, b'@@ '), (211, b'\xab\xcd\x00\x124\x05'),
(230, LIT('a')), (232, LIT('b')), (234, [b'c']), (246, [1, b'z']),
(258, {'foo': b'bar'}),
]
TOKENS = [
(5, KWD(b'begin')), (11, KWD(b'end')), (16, KWD(b'"')), (19, KWD(b'@')),
(21, KWD(b'#')), (23, LIT('a')), (25, LIT('BCD')), (30, LIT('Some_Name')),
(41, LIT('foo_xbaa')), (54, 0), (56, 1), (59, -2), (62, 0.5),
(65, 1.234), (71, b'abc'), (77, b''), (80, b'abc ( def ) ghi'),
(98, b'def \x00 4ghi'), (118, b'bach\\slask'), (132, b'foo\nbaa'),
(143, b'this % is not a comment.'), (170, b'foo\nbaa'), (180, b'foobaa'),
(191, b''), (194, b' '), (199, b'@@ '), (211, b'\xab\xcd\x00\x124\x05'),
(226, KWD(b'func')), (230, LIT('a')), (232, LIT('b')),
(234, KWD(b'{')), (235, b'c'), (238, KWD(b'do*')), (241, KWD(b'}')),
(242, KWD(b'def')), (246, KWD(b'[')), (248, 1), (250, b'z'), (254, KWD(b'!')),
(256, KWD(b']')), (258, KWD(b'<<')), (261, LIT('foo')), (266, b'bar'),
(272, KWD(b'>>'))
]
OBJS = [
(23, LIT('a')), (25, LIT('BCD')), (30, LIT('Some_Name')),
(41, LIT('foo_xbaa')), (54, 0), (56, 1), (59, -2), (62, 0.5),
(65, 1.234), (71, b'abc'), (77, b''), (80, b'abc ( def ) ghi'),
(98, b'def \x00 4ghi'), (118, b'bach\\slask'), (132, b'foo\nbaa'),
(143, b'this % is not a comment.'), (170, b'foo\nbaa'), (180, b'foobaa'),
(191, b''), (194, b' '), (199, b'@@ '), (211, b'\xab\xcd\x00\x124\x05'),
(230, LIT('a')), (232, LIT('b')), (234, [b'c']), (246, [1, b'z']),
(258, {'foo': b'bar'}),
]
class TestPSBaseParser(TestCase):
def get_tokens(self, s):
from io import BytesIO
class MyParser(PSStackParser):
def flush(self):
self.add_results(*self.popall())
@@ -76,8 +68,6 @@ func/a/b{(c)do*}def
return r
def get_objects(self, s):
from io import BytesIO
class MyParser(PSStackParser):
def flush(self):
self.add_results(*self.popall())
@@ -91,18 +81,18 @@ func/a/b{(c)do*}def
return r
def test_1(self):
tokens = self.get_tokens(self.TESTDATA)
tokens = self.get_tokens(TESTDATA)
logging.info(tokens)
assert_equal(tokens, self.TOKENS)
self.assertEqual(tokens, TOKENS)
return
def test_2(self):
objs = self.get_objects(self.TESTDATA)
objs = self.get_objects(TESTDATA)
logging.info(objs)
assert_equal(objs, self.OBJS)
self.assertEqual(objs, OBJS)
return
if __name__ == '__main__':
#import logging,sys,os,six
#logging.basicConfig(level=logging.DEBUG, filename='%s_%d.%d.log'%(os.path.basename(__file__),sys.version_info[0],sys.version_info[1]))
nose.runmodule()
main()
+101
View File
@@ -0,0 +1,101 @@
from os import remove
from tempfile import mktemp
from pdfmajor.converters.writers.yaml import YAMLMaker
# Make temp file
FILE_NAME = mktemp()
with YAMLMaker(FILE_NAME) as yml:
yml.write("name", "bob")
yml.write("age", "20")
with yml.object("wife") as wife:
wife.write("name", "kory")
wife.write("age", "21")
with yml.array("children") as children:
with children.object() as child:
child.write("name", "Mike")
with child.array("toys") as toys:
toys.write("lion")
toys.write("bear")
with yml.object("computer") as computer:
computer.write("type", "awesome")
assert(open(FILE_NAME, 'r').read() == """name: bob
age: 20
wife:
name: kory
age: 21
children:
- name: Mike
toys:
- lion
- bear
computer:
type: awesome""")
with YAMLMaker(FILE_NAME) as yml:
yml.write("name", "bob")
yml.write("age", "20")
yml.place_object("wife", {
"name": "kory",
"age": 21
})
with yml.array("children") as children:
with children.object() as child:
child.write("name", "Mike")
child.place_array("toys", [
"lion", "bear"
])
yml.place_object("computer", {"type": "awesome"})
assert(open(FILE_NAME, 'r').read() == """name: bob
age: 20
wife:
name: kory
age: 21
children:
- name: Mike
toys:
- lion
- bear
computer:
type: awesome""")
with YAMLMaker(FILE_NAME) as yml:
with yml.array('pages') as pages:
with pages.object() as page:
page.write("id", 0)
page.write("width", 100)
page.write("height", 200)
with page.array("children") as children:
with children.object() as curve:
with curve.object("stroke") as fill:
fill.write("type", "DeviceRGB")
fill.write("value", ['1','2','3'])
curve.write("x0", 0)
curve.write("x1", 1)
with curve.object("fill") as fill:
fill.write("type", "DeviceRGB")
fill.write("value", ['1','2','3'])
curve.write("y0", 0)
curve.write("y1", 1)
assert(open(FILE_NAME, 'r').read() == """pages:
- id: 0
width: 100
height: 200
children:
- stroke:
type: DeviceRGB
value: ['1', '2', '3']
x0: 0
x1: 1
fill:
type: DeviceRGB
value: ['1', '2', '3']
y0: 0
y1: 1""")
# Clean up
remove(FILE_NAME)
+4 -5
View File
@@ -52,11 +52,10 @@ def main(args=None):
if len(parsed_args.files) == 1:
if parsed_args.output_file != "-":
func_args['output_file'] = open(parsed_args.output_file, "wb")
with open(parsed_args.files[0], 'rb') as input_file:
convert_file(
input_file,
**func_args
)
convert_file(
parsed_args.files[0],
**func_args
)
else:
print("Multiple files not supported yet.")
return 1