8 Commits
Author SHA1 Message Date
ari 711b67a9d2 - rewrite of interpreter to use more functional-friendly methods
- introducing of concurent processing
- new text/json/xml/html conversions (where all work)
- added several tests for writer modules, conversion and interpreter
2018-12-22 23:32:48 -05:00
ari b8b167e827 - updated docs
- added text conversion
2018-12-22 23:32:11 -05:00
ari 0590d8c8f9 - moved sample folder to tests
- updated tests dep
2018-12-22 22:49:02 -05:00
ari bf46dee422 - renamed convert_sample to test 2018-12-22 22:45:20 -05:00
ari bc20627d9f - custom formatters done
- json formatter works now
- added some tests for writers
2018-12-22 22:44:53 -05:00
ari 3ba0b95001 - moved parser files from old interpeter into parser folder
- renamed interpreter_v2 to just interpreter
- removed legacy code
2018-12-20 10:41:10 -05:00
ari 487f9b6c08 - working intrepreter v2 2018-12-20 10:33:14 -05:00
ari 1d11cd1c7f new interperter 2018-12-19 11:24:02 -05:00
113 changed files with 3357 additions and 3075 deletions

No files matched your search

+1 -2
View File
@@ -9,6 +9,5 @@ tests/*.xml
tests/*.txt
.idea/
.vscode
samples/output
samples/pdfmajor
tests/samples/output
**__pycache**
+1 -1
View File
@@ -1 +1 @@
v1.2.0
v1.3.0
+10 -14
View File
@@ -11,9 +11,7 @@
</tr>
</table>
PDF-Major is a fork of `PDFMiner.six`. It is meant to be a more light-weight implementation that makes fewer assumptions on the data. Additionally, this version intends to be more up to date with recent versions of python and make liberal use of the typing capabilities of python for improved future maintainability.
The ultimate goal is to have a functional-programming with typing fork of the original library.
PDF-Major is a fork of `PDFMiner.six`. It is meant to be a more light-weight implementation that makes fewer assumptions on the data. The ultimate goal of this project is to provide a simpler, faster and more functional library that both casual and low-level developers could build upon.
## Documentation
@@ -21,34 +19,32 @@ See documentation [here](https://asosnovsky.github.io/pdfmajor/).
## Basic Example
```py
from pdfmajor.extractor import extract_items_from_pdf
from pdfmajor.interpreter import PageInterpreter
for page in extract_items_from_pdf('path/to/your/file.pdf'):
print('page-start', page)
for page in PDFInterpreter("/path/to/pdf.pdf"):
print("page start", page.page_num)
for item in page:
print(' ', item)
print('page-end', page)
print(" >", item)
print("page end", page.page_num)
```
## Why Another Fork?
`PDFMiner` was designed to run in python 2.7>, `PDFMiner.six` was written to bring cross-version support for python 2 and 3. This versions intends to make use of the latest improvement in the python language while remaining fairly simple to develop with.
`PDFMiner` was designed to run in python 2.7>, `PDFMiner.six` was written to bring cross-version support for python 2 and 3. However, after attempting to build and extend upon `PDFMiner.six` I found it rather difficult to extend and identify additional details regarding the items encoded in the pdf. This library will attempt to expose as much information as possible to end-users, without having them solely rely on to-xml or to-json exports.
## Features
* Functional extraction method based on generators
* Written entirely in Python.
* Parse, analyze, and convert PDF documents.
* PDF-1.7 specification support. (well, almost)
* Font-Color extraction
* Shape fill and stroke color extraction
* CJK languages and vertical writing scripts support.
* Various font types (Type1, TrueType, Type3, and CID) support.
* Basic encryption (RC4) support.
* Font-Color extraction
* Shape fill and stroke color extraction
**Note**: We took out the layout-analysis process in this version (there is no more LTTextHorizontal or LTTextVertical). While the mathematics behind the grouping process was sound, the coupling of the layout-analysis process with the parsing and interpretation process produced unfriendly-code. This feature could be brought back by extending the `pdfmajor.layouts.PDFLayoutAnalyzer` class, but at the current time is not supported.
**Note**: We took out the layout-analysis process in this version (there is no more LTTextHorizontal or LTTextVertical). While the mathematics behind the grouping process was sound, the coupling of the layout-analysis process with the parsing and interpretation process produced unfriendly-code. This feature could be brought back by running your own implementation of it on the `pdfmajor.interpreter.PageInterpreter` class, but at the current time is not supported.
## How to Install
+54 -103
View File
@@ -2,34 +2,37 @@
The library is constructed from 4 base modules:
* **extractor**: contains a high-level function for functional extraction of pdf-assets
* **interpreter**: contains low-level classes for extracting fundamental data-structures from the documents
* **converters**: contains high-level classes for conversion of the fundamental pdf structures to other formats
* **parser**: contains low-level classes for extracting fundamental data-structures from the documents
* **interpreter**: a interpreter of the pdf-standard commands
* **converters**: contains high-level functions for conversion of the fundamental pdf structures to other formats
* **imagewriter**: contains a simple implementation for converting PDF Image Streams to png/bmp/img formats
* **layouts**: contains an abstraction layer for the data-structures found in the pdf
## extractors
Contains a high-level function for functional extraction of pdf-assets
## parser
WIP
## interpreter
a interpreter of the pdf-standard commands
### Example
```py
from pdfmajor.extractor import extract_items_from_pdf
from pdfmajor.interpreter import PageInterpreter
for page in extract_items_from_pdf('path/to/your/file.pdf'):
print('page-start', page)
for page in PDFInterpreter("/path/to/pdf.pdf"):
print("page start", page.page_num)
for item in page:
print(' ', item)
print('page-end', page)
print(" >", item)
print("page end", page.page_num)
```
### extract_items_from_pdf(input_file_path,maxpages,password,caching,check_extractable,pagenos,debug_level) -> generator[LTPage]
### interpreter.PDFInterpreter
This generator-function yields individual pages which contain their respected items.
#### Arguments
- `input_file_path`: [str](str)
- `input_file_path`: [str](#)
- `preload`: [bool](#) defaults to False
- `maxpages`: [int](#) defaults to 0
- `password`: [str](#) defaults to None
- `caching`: [bool](#) defaults to True
@@ -37,39 +40,57 @@ This generator-function yields individual pages which contain their respected it
- `pagenos`: [List[int]](#) defaults to None
- `debug_level`: [logging.levels](#https://docs.python.org/3/library/logging.html#levels) defaults logging.WARNING
#### Return Value
This function returns a generator that makes `LTPage`
#### Yield Value
This function returns a generator that yields [PDFInterpreter](#interpreterpageinterpreter).
## interpreter
WIP
### interpreter.PageInterpreter
This generator-function-class yields individual [layout items](#layout-items).
### Layout Items
All layout items extend the `LTItem` class. There are two kinds of layout items:
- LTComponent: extends the base `LTItem` class, this class will have additional values such as boundary boxes, height and width
- LTContainer: extends the `LTComponent` class, this class is used to contain elements of the pdf that would have child elements. Iterating on this element will output its child elements.
#### Layout Containers
All of these classes extend the LTContainer class.
- LTXObject: a layout item containing other additional layout items
- LTCharBlock: a layout item containing LTChars
#### Layout Components
All of these classes extend the LTComponent class.
- LTChar: an individual character
- LTCurves: represents a collection of svg-paths (available under `self.paths`)
- LTImage: a component containing information regarding an image
## converters
Contains high-level classes for conversion of the fundamental pdf structures to other formats. This library includes 4 high-level conversion classes:
Contains high-level functions for conversion of the fundamental pdf structures to other formats. This library includes 4 high-level conversion cases:
- HTMLConverter
- JSONConverter
- XMLConverter
- TextConverter
- HTML
- JSON
- XML
- Text
These classes all extend the [PDFConverter](#converters.PDFConverter). To use them simply call the static method [parse_file](#converterspdfconverterparse_fileinput_file-output_file-image_folder_path-codec-maxpages-password-caching-check_extractable-pagenos).
These formats are all generated using the [PageInterpreter](#PDFInterpreter). To use them simply call the static method [parse_file](#converterspdfconverter).
### Example
```py
from pdfmajor.converters import HTMLConverter
from pdfmajor.converters import convert_file
input_file = open('path/to/input/file.pdf', 'rb')
output_file = open('path/to/output/file.html', 'wb')
HTMLConverter.parse_file(
input_file,
output_file
convert_file(
"path/to/input/file.pdf",
"path/to/output/file.html",
out_type="html"
)
input_file.close()
output_file.close()
```
### converters.convert_file(input_file, output_file, image_folder_path, codec, maxpages, password, caching, check_extractable, pagenos, out_type)
### converters.convert_file
A high-level abstraction for the conversion classes.
@@ -84,76 +105,6 @@ A high-level abstraction for the conversion classes.
- `pagenos`: [List[int]](#) defaults to None
- `out_type`: [str](#) defaults to 'html'
### Example (converters.convert_file)
```py
from pdfmajor.converters import convert_file
input_file = open('path/to/input/file.pdf', 'rb')
output_file = open('path/to/output/file.html', 'wb')
convert_file(
input_file,
output_file
)
input_file.close()
output_file.close()
```
### converters.PDFConverter
Extends `layouts.PDFLayoutAnalyzer`.
#### converters.PDFConverter.\_\_init\_\_(rsrcmgr, outfp, imagewriter, codec, pageno)
- `rsrcmgr`: [PDFResourceManager](#interpreter.PDFResourceManager)
- `outfp`: [TextIOWrapper](https://docs.python.org/3/library/io.html#io.TextIOWrapper)
- `imagewriter`: [ImageWriter](#imagewriter.ImageWriter)
- `codec`: [str](#) defaults to 'utf-8'
- `pageno`: [int](#) defaults to 1
initialization function for the class
#### converters.PDFConverter.parse_file(input_file, output_file, image_folder_path, codec, maxpages, password, caching, check_extractable, pagenos)
returns the output file [TextIOWrapper](https://docs.python.org/3/library/io.html#io.TextIOWrapper).
- `input_file`: [TextIOWrapper](https://docs.python.org/3/library/io.html#io.TextIOWrapper)
- `output_file`: [TextIOWrapper](https://docs.python.org/3/library/io.html#io.TextIOWrapper)
- `image_folder_path`: [str](#) defaults to None
- `codec`: [str](#) defaults to 'utf-8'
- `maxpages`: [int](#) defaults to 0
- `password`: [str](#) defaults to None
- `caching`: [bool](#) defaults to True
- `check_extractable`: [bool](#) defaults to True
- `pagenos`: [List[int]](#) defaults to None
## imagewriter
WIP
## layouts
WIP: this documentation set is not yet complete.
### Layout Items
All layout items extend the `LTItem` class. There are two kinds of layout items:
- LTComponent: extends the base `LTItem` class, this class will have additional values such as boundary boxes, height and width
- LTContainer: extends the `LTComponent` class, this class is used to contain elements of the pdf that would have child elements. Iterating on this element will output its child elements.
#### Layout Containers
All of these classes extend the LTContainer class.
- LTPage: a layout item containing all elements of the given page.
- LTFigure: a layout item containing LTCurves and LTImages
- LTCharBlock: a layout item containing LTChars
#### Layout Components
All of these classes extend the LTComponent class.
- LTChar: an individual character
- LTCurves: represents a collection of svg-paths (available under `self.paths`)
- LTImage: a component containing information regarding an image
+10 -14
View File
@@ -11,9 +11,7 @@
</tr>
</table>
PDF-Major is a fork of `PDFMiner.six`. It is meant to be a more light-weight implementation that makes fewer assumptions on the data. Additionally, this version intends to be more up to date with recent versions of python and make liberal use of the typing capabilities of python for improved future maintainability.
The ultimate goal is to have a functional-programming with typing fork of the original library.
PDF-Major is a fork of `PDFMiner.six`. It is meant to be a more light-weight implementation that makes fewer assumptions on the data. The ultimate goal of this project is to provide a simpler, faster and more functional library that both casual and low-level developers could build upon.
## Documentation
@@ -21,34 +19,32 @@ See documentation [here](https://asosnovsky.github.io/pdfmajor/).
## Basic Example
```py
from pdfmajor.extractor import extract_items_from_pdf
from pdfmajor.interpreter import PageInterpreter
for page in extract_items_from_pdf('path/to/your/file.pdf'):
print('page-start', page)
for page in PDFInterpreter("/path/to/pdf.pdf"):
print("page start", page.page_num)
for item in page:
print(' ', item)
print('page-end', page)
print(" >", item)
print("page end", page.page_num)
```
## Why Another Fork?
`PDFMiner` was designed to run in python 2.7>, `PDFMiner.six` was written to bring cross-version support for python 2 and 3. This versions intends to make use of the latest improvement in the python language while remaining fairly simple to develop with.
`PDFMiner` was designed to run in python 2.7>, `PDFMiner.six` was written to bring cross-version support for python 2 and 3. However, after attempting to build and extend upon `PDFMiner.six` I found it rather difficult to extend and identify additional details regarding the items encoded in the pdf. This library will attempt to expose as much information as possible to end-users, without having them solely rely on to-xml or to-json exports.
## Features
* Functional extraction method based on generators
* Written entirely in Python.
* Parse, analyze, and convert PDF documents.
* PDF-1.7 specification support. (well, almost)
* Font-Color extraction
* Shape fill and stroke color extraction
* CJK languages and vertical writing scripts support.
* Various font types (Type1, TrueType, Type3, and CID) support.
* Basic encryption (RC4) support.
* Font-Color extraction
* Shape fill and stroke color extraction
**Note**: We took out the layout-analysis process in this version (there is no more LTTextHorizontal or LTTextVertical). While the mathematics behind the grouping process was sound, the coupling of the layout-analysis process with the parsing and interpretation process produced unfriendly-code. This feature could be brought back by extending the `pdfmajor.layouts.PDFLayoutAnalyzer` class, but at the current time is not supported.
**Note**: We took out the layout-analysis process in this version (there is no more LTTextHorizontal or LTTextVertical). While the mathematics behind the grouping process was sound, the coupling of the layout-analysis process with the parsing and interpretation process produced unfriendly-code. This feature could be brought back by running your own implementation of it on the `pdfmajor.interpreter.PageInterpreter` class, but at the current time is not supported.
## How to Install
+11 -15
View File
@@ -16,9 +16,7 @@
</tr>
</table>
PDF-Major is a fork of `PDFMiner.six`. It is meant to be a more light-weight implementation that makes fewer assumptions on the data. Additionally, this version intends to be more up to date with recent versions of python and make liberal use of the typing capabilities of python for improved future maintainability.
The ultimate goal is to have a functional-programming with typing fork of the original library.
PDF-Major is a fork of `PDFMiner.six`. It is meant to be a more light-weight implementation that makes fewer assumptions on the data. The ultimate goal of this project is to provide a simpler, faster and more functional library that both casual and low-level developers could build upon.
## Documentation
@@ -26,34 +24,32 @@ See documentation [here](https://asosnovsky.github.io/pdfmajor/).
## Basic Example
```py
from pdfmajor.extractor import extract_items_from_pdf
from pdfmajor.interpreter import PageInterpreter
for page in extract_items_from_pdf('path/to/your/file.pdf'):
print('page-start', page)
for page in PDFInterpreter("/path/to/pdf.pdf"):
print("page start", page.page_num)
for item in page:
print(' ', item)
print('page-end', page)
print(" >", item)
print("page end", page.page_num)
```
## Why Another Fork?
`PDFMiner` was designed to run in python 2.7>, `PDFMiner.six` was written to bring cross-version support for python 2 and 3. This versions intends to make use of the latest improvement in the python language while remaining fairly simple to develop with.
`PDFMiner` was designed to run in python 2.7>, `PDFMiner.six` was written to bring cross-version support for python 2 and 3. However, after attempting to build and extend upon `PDFMiner.six` I found it rather difficult to extend and identify additional details regarding the items encoded in the pdf. This library will attempt to expose as much information as possible to end-users, without having them solely rely on to-xml or to-json exports.
## Features
* Functional extraction method based on generators
* Written entirely in Python.
* Parse, analyze, and convert PDF documents.
* PDF-1.7 specification support. (well, almost)
* Font-Color extraction
* Shape fill and stroke color extraction
* CJK languages and vertical writing scripts support.
* Various font types (Type1, TrueType, Type3, and CID) support.
* Basic encryption (RC4) support.
* Font-Color extraction
* Shape fill and stroke color extraction
**Note**: We took out the layout-analysis process in this version (there is no more LTTextHorizontal or LTTextVertical). While the mathematics behind the grouping process was sound, the coupling of the layout-analysis process with the parsing and interpretation process produced unfriendly-code. This feature could be brought back by extending the `pdfmajor.layouts.PDFLayoutAnalyzer` class, but at the current time is not supported.
**Note**: We took out the layout-analysis process in this version (there is no more LTTextHorizontal or LTTextVertical). While the mathematics behind the grouping process was sound, the coupling of the layout-analysis process with the parsing and interpretation process produced unfriendly-code. This feature could be brought back by running your own implementation of it on the `pdfmajor.interpreter.PageInterpreter` class, but at the current time is not supported.
## How to Install
@@ -101,7 +97,7 @@ OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
"""
__version__ = (1, 2, 0)
__version__ = (1, 3, 0)
if __name__ == '__main__':
print(
-246
View File
@@ -1,246 +0,0 @@
import logging
from typing import Dict
from contextlib import contextmanager
from ..layouts import LTPage
from ..layouts import LTItem
from ..layouts import LTCurve, LTLine, LTRect, CurvePath
from ..layouts import LTFigure
from ..layouts import LTImage
from ..layouts import LTChar, LTCharBlock
from ..utils import enc
from .PDFConverter import PDFConverter
from ..interpreter.PDFGraphicState import PDFGraphicStateColor
log = logging.getLogger(__name__)
def get_color(col: PDFGraphicStateColor):
color_type, color_val = col.get_color()
if color_type is not None:
if color_type == 'rgb':
return f"rgb({','.join(map(str, color_val))})"
elif color_type == 'cmyk':
return f"cmyk({','.join(map(str, color_val))})"
elif color_type == 'gray':
return f'rgba(0,0,0,{1-color_val[0]})'
elif color_type == "custom":
# Todo: convert these to more friendly color types
return f"custom({col.custom['type']})"
return "none"
class HTMLConverter(PDFConverter):
def __init__(self,
rsrcmgr,
outfp,
codec='utf-8',
pageno=1,
imagewriter=None
):
PDFConverter.__init__(self, rsrcmgr, outfp, imagewriter=imagewriter, codec=codec, pageno=pageno)
self.__levels_deep = 0
self.__current_height = None
self.__current_width = None
self.write_header()
def write(self, text: str, lineend: str = '\n', deep_space: str = ' '):
text = deep_space*self.__levels_deep + text + lineend
self.write_raw(text)
def write_raw(self, text):
if self.codec:
text = text.encode(self.codec)
self.outfp.write(text)
def write_header(self):
self.write('<html>')
self.__levels_deep += 1
self.write('<head>')
self.__levels_deep += 1
meta_attr = {
'http-equiv': "Content-Type",
'content': "text/html",
}
if self.codec:
meta_attr['charset'] = self.codec
self.place_elm_close('meta', attr_dict=meta_attr, as_childless=False)
with self.place_elm_with_child('style'):
self.write('''.page { position: relative; overflow: hidden;border: 1px black solid;}''')
self.__levels_deep -= 1
self.write('</head>')
self.write('<body>')
self.__levels_deep += 1
def write_footer(self):
self.write('</body>')
self.__levels_deep -= 1
self.write('</html>')
self.__levels_deep -= 1
@contextmanager
def place_elm_with_child(self, tag_name: str, attr_dict: Dict[str, str] = {}, css: Dict[str, str] = None, no_additional_char = False):
lineend = '\n'
deep_space = ' '
if no_additional_char:
lineend = ''
deep_space = ''
if css is not None:
attr_dict['style'] = ";".join([
f'{name}: {value}'
for name, value in css.items()
])
attrs = " ".join([
f'{name}="{value}"'
for name, value in attr_dict.items()
])
self.write(f"<{tag_name} {attrs}>", lineend=lineend)
self.__levels_deep += 1
yield
self.__levels_deep -= 1
self.write(f"</{tag_name}>", deep_space=deep_space)
def place_elm_close(self, tag_name: str, attr_dict: Dict[str, str] = {}, css: Dict[str, str] = None, as_childless = True):
if css is not None:
attr_dict['style'] = ";".join([
f'{name}: {value}'
for name, value in css.items()
])
attrs = " ".join([
f'{name}="{value}"'
for name, value in attr_dict.items()
])
if as_childless:
self.write(f"<{tag_name} {attrs}></{tag_name}>")
else:
self.write(f"<{tag_name} {attrs}/>")
def place_image(self, item: LTImage):
name = item.name
if self.imagewriter is not None:
name = self.imagewriter.export_image(item)
self.place_elm_close('img', {
'src': enc(name, None),
'width': f"{item.width}px",
'height': f"{item.height}px",
})
return
def render_curve(self, item: LTCurve):
css = {
'position': 'absolute',
'left': f'{item.x0}px',
'bottom': f'{item.y0}px',
'width': f'{item.width}px',
'height': f'{item.height}px',
}
path = ""
svg_attr = {
'class': 'curve',
'z-index': 2
}
p_vals = { "x": [], "y": [] }
for p in item.paths:
for pt in p.points:
p_vals['x'].append(pt.x)
p_vals['y'].append(pt.y)
mx_x = max([*p_vals['x'], 1])
mn_x = min([*p_vals['x'], 0])
mx_y = max([*p_vals['y'], 1])
mn_y = min([*p_vals['y'], 0])
if mx_x == mn_x:
mn_x -= 1
if mx_y == mn_y:
mn_y -= 1
dx = 1/( mx_x - mn_x )
dy = (item.height/item.width)/( mx_y - mn_y )
ROUND_VAL = 5
svg_attr.update({
# "viewBox": f"{min(p_vals['x'])} {min(p_vals['y'])} {max(p_vals['x'])} {max(p_vals['y'])}",
"viewBox": f"{round(mn_x*dx, ROUND_VAL)} {round(mn_y*dy, ROUND_VAL)} {round(mx_x*dx , ROUND_VAL)} {round(mx_y*dy, ROUND_VAL)}",
})
for p in item.paths:
if p.method == CurvePath.METHOD.MOVE_TO:
path += f'M{" ".join( f"{round(dx*pt.x, ROUND_VAL)} {round(dy*pt.y, ROUND_VAL)}" for pt in p.points )} '
if p.method == CurvePath.METHOD.LINE_TO:
path += f'L{" ".join( f"{round(dx*pt.x, ROUND_VAL)} {round(dy*pt.y, ROUND_VAL)}" for pt in p.points )} '
if p.method == CurvePath.METHOD.CLOSE_PATH:
path += 'Z'
with self.place_elm_with_child('svg', svg_attr, css):
self.place_elm_close('path', {
"d": path,
'stroke': get_color(item.stroke),
'fill': get_color(item.fill),
})
def receive_layout(self, ltpage: LTPage):
def render(item: LTItem):
if isinstance(item, LTPage):
with self.place_elm_with_child('div', { "class": 'page', "id": f"page-{item.pageid}" }, { 'width': f'{item.width}px', 'height': f'{item.height}px' }):
self.__current_height = item.height
self.__current_width = item.width
for child in item:
render(child)
elif isinstance(item, LTCurve):
self.render_curve(item)
elif isinstance(item, LTFigure):
with self.place_elm_with_child('div', { "class": 'figure' }, {
'position': 'absolute',
'left': f'{item.x0}px',
'bottom': f'{item.y0}px',
'width': f'{item.width}px',
'height': f'{item.height}px',
}):
for child in item:
render(child)
elif isinstance(item, LTImage):
self.place_image(item)
elif isinstance(item, LTChar):
with self.place_elm_with_child('span', { 'class': 'char' }, {
'width': item.width,
'min-width': item.width,
'max-width': item.width,
'height': item.height,
'min-height': item.height,
'max-height': item.height,
}):
self.write(item.get_text())
elif isinstance(item, LTCharBlock):
with self.place_elm_with_child('span', { 'class': 'char-block' }, {
'position': 'absolute',
'left': f'{item.x0}',
'bottom': f'{item.y0}',
'text-align': 'justify',
'font-size': f'{item.size}px',
'font-family': item.fontname,
'font-weight': item.font.font_weight,
'color': get_color(item.graphicstate.ncolor),
'width': item.width,
'min-width': item.width,
'max-width': item.width,
'height': item.height,
'min-height': item.height,
'max-height': item.height,
'transform': f'skew({item.font.italic_angle}deg, 0deg)',
'text-align': 'center',
'display': 'flex',
'flex-flow': 'row nowrap',
'justify-content': 'space-between',
# 'padding-left': item.font.leading
}):
for char in item:
render(char)
render(ltpage)
def close(self):
self.write_footer()
-183
View File
@@ -1,183 +0,0 @@
import logging
from typing import Dict, Union
from io import BytesIO
from contextlib import contextmanager
from ..layouts import LTPage
from ..layouts import LTItem
from ..layouts import LTCurve
from ..layouts import LTLine
from ..layouts import LTRect
from ..layouts import LTFigure
from ..layouts import LTImage
from ..layouts import LTChar, LTCharBlock
from ..utils import enc
from .PDFConverter import PDFConverter
from ..interpreter.PDFGraphicState import PDFGraphicStateColor
log = logging.getLogger(__name__)
def get_color(col: PDFGraphicStateColor):
color_type, color_val = col.get_color()
if color_type is not None:
if color_type == 'rgb':
return f"rgb({','.join(map(str, color_val))})"
elif color_type == 'cmyk':
return f"cmyk({','.join(map(str, color_val))})"
elif color_type == 'gray':
return f'gray({color_val})'
return ""
class JSONConverter(PDFConverter):
def __init__(self,
rsrcmgr,
outfp: BytesIO,
codec: str='utf-8',
pageno: int=1,
imagewriter=None,
):
PDFConverter.__init__(self, rsrcmgr, outfp, imagewriter=imagewriter, codec=codec, pageno=pageno)
self.__last_levels_deep:int = None
self.__levels_deep = 0
self.write_header()
@property
def _levels_deep(self): return self.__levels_deep
@_levels_deep.setter
def _levels_deep(self, val: int):
self.__last_levels_deep = self.__levels_deep
self.__levels_deep = val
def write(self, text: str):
if self.__last_levels_deep == self._levels_deep:
text = ',' + text
self.write_raw(text)
def write_raw(self, text):
if self.codec:
text = text.encode(self.codec)
self.outfp.write(text)
self._levels_deep = self._levels_deep
def write_header(self):
self.write("{")
self._levels_deep += 1
if self.codec:
self.write('"encoding": "%s"' % self.codec)
self.write('"pages": [')
self._levels_deep += 1
def write_footer(self):
self._levels_deep -= 1
self.write(']')
self._levels_deep -= 1
self.write('}')
@contextmanager
def place_elm_with_child(self, tag_name: str, attr_dict: Dict[str, str] = {}):
attrs = ", ".join([
f'"{name}": "{value}"'
for name, value in attr_dict.items()
])
self.write('{ "type": "%s", "attrs": { %s } , "children": [' % (tag_name, attrs) )
self._levels_deep += 1
yield
self._levels_deep -= 1
self.write("]}")
def place_elm_close(self, tag_name: str, attr_dict: Dict[str, str] = {}):
attrs = ", ".join([
f'"{name}": "{value}"'
for name, value in attr_dict.items()
])
self.write('{ "type": "%s", "attrs": { %s } }' % (tag_name, attrs) )
def place_image(self, item: LTImage):
name = item.name
if self.imagewriter is not None:
name = self.imagewriter.export_image(item)
self.place_elm_close('img', {
'src': enc(name, None),
"x0": item.x0,
"x1": item.x1,
"y0": item.y0,
"y1": item.y1,
})
def render_curve(self, item: LTCurve):
attr = {
"x0": item.x0,
"x1": item.x1,
"y0": item.y0,
"y1": item.y1,
"stroke-color": get_color(item.stroke),
"fill-color": get_color(item.fill),
}
if isinstance(item, LTRect):
return self.place_elm_close('rect', attr)
elif isinstance(item, LTLine):
return self.place_elm_close('line', attr)
with self.place_elm_with_child('curve', attr):
for path in item.paths:
with self.place_elm_with_child('path', { "type": path.method.name }):
for point in path.points:
self.place_elm_close('point', { 'x': point.x, 'y': point.y })
def receive_layout(self, ltpage: LTPage):
def render(item: LTItem):
if isinstance(item, LTPage):
with self.place_elm_with_child('page', {
"width": item.width,
"height": item.height
}):
for child in item:
render(child)
elif isinstance(item, LTCurve):
self.render_curve(item)
elif isinstance(item, LTFigure):
with self.place_elm_with_child('figure', {
"x0": item.x0,
"x1": item.x1,
"y0": item.y0,
"y1": item.y1,
}):
for child in item:
render(child)
elif isinstance(item, LTImage):
self.place_image(item)
elif isinstance(item, LTChar):
self.place_elm_close('char', {
'font-size': item.size,
'font-family': item.fontname,
"x0": item.x0,
"x1": item.x1,
"y0": item.y0,
"y1": item.y1,
"stroke-color": get_color(item.graphicstate.scolor),
"fill-color": get_color(item.graphicstate.ncolor),
"text": item.get_text()
})
elif isinstance(item, LTCharBlock):
self.place_elm_close('char-block', {
'font-size': item.size,
'font-family': item.fontname,
"x0": item.x0,
"x1": item.x1,
"y0": item.y0,
"y1": item.y1,
"stroke-color": get_color(item.graphicstate.scolor),
"fill-color": get_color(item.graphicstate.ncolor),
"text": item.get_text()
})
render(ltpage)
def close(self):
self.write_footer()
-87
View File
@@ -1,87 +0,0 @@
import re
import os
from typing import List
from io import BytesIO, StringIO, TextIOWrapper
from tqdm import tqdm
from ..layouts import PDFLayoutAnalyzer
from ..interpreter.PDFResourceManager import PDFResourceManager
from ..interpreter.PDFPage import PDFPage
from ..interpreter.PDFPageInterpreter import PDFPageInterpreter
from ..imagewriter import ImageWriter
from ..utils import set_log_level, get_logger, logging
log = get_logger(__name__)
## PDFConverter
##
class PDFConverter(PDFLayoutAnalyzer):
@classmethod
def parse_file(cls,
input_file: TextIOWrapper,
output_file: TextIOWrapper,
image_folder_path: str = None,
codec: str = 'utf-8',
maxpages: int = 0,
password: str = None,
caching: bool = True,
check_extractable: bool = True,
pagenos: List[int] = None,
dont_export_images: bool = False,
debug_level: int = logging.WARNING,
) -> TextIOWrapper:
set_log_level(debug_level)
if dont_export_images:
image_folder_path = None
if not dont_export_images and image_folder_path is None and output_file.name is not None:
image_folder_path = os.path.dirname(output_file.name)
log.debug("Creating resources....")
rsrcmgr = PDFResourceManager(caching=caching)
imagewriter=ImageWriter(image_folder_path)
device = cls(
rsrcmgr,
output_file,
codec=codec,
imagewriter=imagewriter
)
interpreter = PDFPageInterpreter(rsrcmgr, device)
log.debug("Reading pages....")
pages = PDFPage.get_pages(input_file, pagenos=pagenos, maxpages=maxpages, password=password, caching=caching, check_extractable=check_extractable)
page_waiter = tqdm(enumerate(pages), disable=debug_level > logging.INFO)
for _, page in page_waiter:
page_waiter.set_description("Processing pdf...")
interpreter.process_page(page)
page_waiter.close()
device.close()
return output_file
def __init__(self, rsrcmgr: PDFResourceManager, outfp: TextIOWrapper, imagewriter: ImageWriter, codec: str='utf-8', pageno: int=1):
PDFLayoutAnalyzer.__init__(self, rsrcmgr, pageno=pageno)
self.imagewriter = imagewriter
self.outfp = outfp
self.codec = codec
if hasattr(self.outfp, 'mode'):
if 'b' in self.outfp.mode:
self.outfp_binary = True
else:
self.outfp_binary = False
else:
if isinstance(self.outfp, BytesIO):
self.outfp_binary = True
elif isinstance(self.outfp, StringIO):
self.outfp_binary = False
else:
try:
self.outfp.write(u"é")
self.outfp_binary = False
except TypeError:
self.outfp_binary = True
return
-52
View File
@@ -1,52 +0,0 @@
import logging
from typing import Dict
from contextlib import contextmanager
from ..layouts import LTPage
from ..layouts import LTItem
from ..layouts import LTCurve
from ..layouts import LTLine
from ..layouts import LTRect
from ..layouts import LTFigure
from ..layouts import LTImage
from ..layouts import LTChar, LTCharBlock
from ..interpreter.PDFGraphicState import PDFGraphicStateColor
from ..utils import enc
from .PDFConverter import PDFConverter
log = logging.getLogger(__name__)
class TextConverter(PDFConverter):
def __init__(self,
rsrcmgr,
outfp,
codec='utf-8',
pageno=1,
imagewriter=None,
):
PDFConverter.__init__(self, rsrcmgr, outfp, imagewriter=imagewriter, codec=codec, pageno=pageno)
def write(self, text: str):
if self.codec:
text = text.encode(self.codec)
self.outfp.write(text)
def place_text(self, char: LTChar):
self.write(char.get_text())
def receive_layout(self, ltpage: LTPage):
def render(item: LTItem):
if isinstance(item, LTPage):
self.write(f'\n-----[Page {self.pageno-1}]-----\n')
for child in item:
render(child)
self.write(f'\n-----[Page {self.pageno-1}]-----\n')
elif isinstance(item, LTFigure):
for child in item:
render(child)
elif isinstance(item, LTChar) or isinstance(item, LTCharBlock):
self.place_text(item)
render(ltpage)
-192
View File
@@ -1,192 +0,0 @@
import logging
import re
import json
from typing import Dict
from xml.sax import saxutils
from contextlib import contextmanager
from ..layouts import LTPage
from ..layouts import LTItem
from ..layouts import LTCurve
from ..layouts import LTLine
from ..layouts import LTRect
from ..layouts import LTFigure
from ..layouts import LTImage
from ..layouts import LTChar, LTCharBlock
from ..interpreter.PDFGraphicState import PDFGraphicStateColor
from ..utils import enc
from .PDFConverter import PDFConverter
log = logging.getLogger(__name__)
def get_color(col: PDFGraphicStateColor):
color_type, color_val = col.get_color()
if color_type is not None:
if color_type == 'rgb':
return f"rgb({','.join(map(str, color_val))})"
elif color_type == 'cmyk':
return f"cmyk({','.join(map(str, color_val))})"
elif color_type == 'gray':
return f'gray({color_val})'
elif color_type == "custom":
# Todo: convert these to more friendly color types
return f"{col.custom['type']}({col.custom['defintion']})"
return ""
FIRST_CAP_RE = re.compile('(.)([A-Z][a-z]+)')
ALL_CAP_RE = re.compile('([a-z0-9])([A-Z])')
def convert_capcase_to_snakecase(text: str) -> str:
s1 = FIRST_CAP_RE.sub(r'\1-\2', text)
return ALL_CAP_RE.sub(r'\1-\2', s1).lower()
class XMLConverter(PDFConverter):
def __init__(self,
rsrcmgr,
outfp,
codec='utf-8',
pageno=1,
imagewriter=None,
):
PDFConverter.__init__(self, rsrcmgr, outfp, imagewriter=imagewriter, codec=codec, pageno=pageno)
self.__levels_deep = 0
self.write_header()
def write(self, text: str, lineend: str = '\n', deep_space: str = ' '):
text = deep_space*self.__levels_deep + text + lineend
self.write_raw(text)
def write_raw(self, text):
if self.codec:
text = text.encode(self.codec)
self.outfp.write(text)
def write_header(self):
if self.codec:
self.write('<?xml version="1.0" encoding="%s" ?>' % self.codec)
else:
self.write('<?xml version="1.0" ?>')
self.write('<pages>')
self.__levels_deep += 1
def write_footer(self):
self.write('</pages>')
@contextmanager
def place_elm_with_child(self, tag_name: str, attr_dict: Dict[str, str] = {}, no_additional_char = False):
lineend = '\n'
deep_space = ' '
if no_additional_char:
lineend = ''
deep_space = ''
attrs = " ".join([
f'{name}="{saxutils.escape(str(value))}"'
for name, value in attr_dict.items()
])
self.write(f"<{tag_name} {attrs}>", lineend=lineend)
self.__levels_deep += 1
yield
self.__levels_deep -= 1
self.write(f"</{tag_name}>", deep_space=deep_space)
def place_elm_close(self, tag_name: str, attr_dict: Dict[str, str] = {}, as_childless = True):
attrs = " ".join([
f'{name}="{saxutils.escape(str(value))}"'
for name, value in attr_dict.items()
])
if as_childless:
self.write(f"<{tag_name} {attrs}></{tag_name}>")
else:
self.write(f"<{tag_name} {attrs}/>")
def place_full_container(self, class_name: str, item: LTItem):
return self.place_elm_with_child(class_name, {
"x0": item.x0,
"x1": item.x1,
"y0": item.y0,
"y1": item.y1,
})
def place_image(self, item: LTImage):
name = item.name
if self.imagewriter is not None:
name = self.imagewriter.export_image(item)
self.place_elm_close('img', {
'src': enc(name, None),
"x0": item.x0,
"x1": item.x1,
"y0": item.y0,
"y1": item.y1,
})
def place_text(self, char: LTChar):
with self.place_elm_with_child('char', {
'x0': char.x0,
'x1': char.x1,
'y0': char.y0,
'y1': char.y1,
}, no_additional_char=True):
self.write(saxutils.escape(char.get_text()), lineend='', deep_space="")
def render_char_block(self, char_block: LTCharBlock):
attr = {
'size': char_block.size,
"color": get_color(char_block.graphicstate.ncolor),
"x0": char_block.x0,
"x1": char_block.x1,
"y0": char_block.y0,
"y1": char_block.y1,
}
for key, value in char_block.font.descriptor.items():
if key != "Type" and "FontFile" not in key:
attr[convert_capcase_to_snakecase(key)] = value
with self.place_elm_with_child("char-block", attr):
for char in char_block:
self.place_text(char)
def render_curve(self, item: LTCurve):
attr = {
"x0": item.x0,
"x1": item.x1,
"y0": item.y0,
"y1": item.y1,
"stroke-color": get_color(item.stroke),
"fill-color": get_color(item.fill),
}
if isinstance(item, LTRect):
return self.place_elm_close('rect', attr)
elif isinstance(item, LTLine):
return self.place_elm_close('line', attr)
with self.place_elm_with_child('curve', attr):
for path in item.paths:
with self.place_elm_with_child('path', { "type": path.method.name }):
for point in path.points:
self.place_elm_close('point', { 'x': point.x, 'y': point.y })
def receive_layout(self, ltpage: LTPage):
def render(item: LTItem):
if isinstance(item, LTPage):
with self.place_full_container('page', item):
for child in item:
render(child)
elif isinstance(item, LTCurve):
self.render_curve(item)
elif isinstance(item, LTFigure):
with self.place_full_container('figure', item):
for child in item:
render(child)
elif isinstance(item, LTImage):
self.place_image(item)
elif isinstance(item, LTCharBlock):
self.render_char_block(item)
render(ltpage)
def close(self):
self.write_footer()
+25 -28
View File
@@ -4,15 +4,15 @@ import re
from typing import List
from .HTMLConverter import HTMLConverter
from .JSONConverter import JSONConverter
from .XMLConverter import XMLConverter
from .TextConverter import TextConverter
from .html import convert_to_html
from .xml import convert_to_xml
from .json import convert_to_json
from .text import convert_to_text
from ..utils import logging
def convert_file(
input_file: io.TextIOWrapper,
output_file: io.TextIOWrapper = None,
input_file: str,
output_file: str = None,
image_folder_path: str = None,
codec: str = 'utf-8',
maxpages: int = 0,
@@ -25,35 +25,32 @@ def convert_file(
out_type: str = 'html',
):
if output_file is None:
output_file = open(
os.path.join(
os.path.dirname(input_file.name),
os.path.basename(
re.sub(r"\.\w+$", "."+out_type, input_file.name)
)
),
'wb'
output_file = os.path.join(
os.path.dirname(input_file.name),
os.path.basename(
re.sub(r"\.\w+$", "."+out_type, input_file.name)
)
)
if re.search(re.escape(out_type) + r'$', output_file.name ) is None:
if re.search(re.escape(out_type) + r'$', output_file ) is None:
raise Exception("Please make sure that the file name and output type match!")
if out_type == 'html':
return HTMLConverter.parse_file(
input_file=input_file,
output_file=output_file,
return convert_to_html(
input_file_path=input_file,
output_file_path=output_file,
image_folder_path=image_folder_path,
dont_export_images=dont_export_images,
codec=codec,
maxpages=maxpages,
password=password,
caching=caching,
check_extractable=check_extractable,
pagenos=pagenos,
dont_export_images=dont_export_images,
debug_level=debug_level,
)
elif out_type == 'xml':
return XMLConverter.parse_file(
input_file=input_file,
output_file=output_file,
return convert_to_xml(
input_file_path=input_file,
output_file_path=output_file,
image_folder_path=image_folder_path,
codec=codec,
maxpages=maxpages,
@@ -65,9 +62,9 @@ def convert_file(
debug_level=debug_level,
)
elif out_type == 'json':
return JSONConverter.parse_file(
input_file=input_file,
output_file=output_file,
return convert_to_json(
input_file_path=input_file,
output_file_path=output_file,
image_folder_path=image_folder_path,
codec=codec,
maxpages=maxpages,
@@ -79,9 +76,9 @@ def convert_file(
debug_level=debug_level,
)
elif out_type == 'text':
return TextConverter.parse_file(
input_file=input_file,
output_file=output_file,
return convert_to_text(
input_file_path=input_file,
output_file_path=output_file,
image_folder_path=image_folder_path,
codec=codec,
maxpages=maxpages,
+189
View File
@@ -0,0 +1,189 @@
from typing import List
from .writers.html import HTMLMaker
from ..interpreter import PDFInterpreter, PageInterpreter, logging
from ..interpreter import LTImage, LTCharBlock, LTChar, LTCurve, LTXObject
from ..interpreter.commands import LTItem
from ..interpreter.commands.state import CurvePath, PDFColor
def convert_to_html(
input_file_path: str,
output_file_path: str,
image_folder_path: str = None,
dont_export_images: bool = False,
codec: str = 'utf-8',
maxpages: int = 0,
password: str = None,
caching: bool = True,
check_extractable: bool = True,
pagenos: List[int] = None,
debug_level: int = logging.WARNING,
):
intepreter = PDFInterpreter(input_file_path,
maxpages=maxpages,
password=password,
caching=caching,
check_extractable=check_extractable,
pagenos=pagenos,
debug_level=debug_level
)
with HTMLMaker(output_file_path, codec=codec) as html:
with html.elm("html", nolineend=True):
with html.elm("head"):
meta_attr = {
'http-equiv': "Content-Type",
'content': "text/html",
}
if html.codec:
meta_attr['charset'] = html.codec
html.singleton("meta", meta_attr)
with html.elm('style'):
html.write('''.page { position: relative; overflow: hidden;border: 1px black solid;}''')
with html.elm("body"):
for page in intepreter:
render_page(html, page)
def render_page(html: HTMLMaker, ltpage: PageInterpreter):
def render(item: LTItem):
if isinstance(item, LTCurve):
render_curve(html, item)
elif isinstance(item, LTXObject):
with html.elm('div', { "class": 'xobject' }, {
'position': 'absolute',
'left': f'{item.x0}px',
'bottom': f'{item.y0}px',
'width': f'{item.width}px',
'height': f'{item.height}px',
}):
for child in item:
render(child)
elif isinstance(item, LTImage):
name = item.name
# if html.imagewriter is not None:
# name = html.imagewriter.export_image(item)
html.singleton('img', attrs={
'src': name,
'background': 'red',
'width': f"{item.width}px",
'height': f"{item.height}px",
})
elif isinstance(item, LTChar):
with html.elm('span', { 'class': 'char' }, {
'width': item.width,
'min-width': item.width,
'max-width': item.width,
'height': item.height,
'min-height': item.height,
'max-height': item.height,
}):
html.write(item.get_text())
elif isinstance(item, LTCharBlock):
with html.elm('span', { 'class': 'char-block' }, {
'position': 'absolute',
'left': f'{item.x0}',
'bottom': f'{item.y0}',
'font-size': f'{item.size}px',
'font-family': item.fontname,
'font-weight': item.font.font_weight,
'color': get_color(item.color, default="black"),
'width': item.width,
'min-width': item.width,
'max-width': item.width,
'height': item.height,
'min-height': item.height,
'max-height': item.height,
'transform': f'skew({item.font.italic_angle}deg, 0deg)',
'text-align': 'center',
'display': 'flex',
'flex-flow': 'row nowrap',
'justify-content': 'space-between',
# 'padding-left': item.font.leading
}):
for char in item:
render(char)
with html.elm('div', { "class": 'page', "id": f"page-{ltpage.page_num}" }, {
'width': f'{ltpage.width}px',
'height': f'{ltpage.height}px'
}):
for child in ltpage:
render(child)
def get_color(col: PDFColor, default: str = ""):
if col.color_space is not None:
if col.color_space.ncomponents == 1:
if type(col.values[0]) in [float, int]:
return f"rgba(0,0,0,{1 - float(col.values[0])})"
elif col.color_space.ncomponents == 3:
return f"rgb({','.join(map(str, col.values))})"
elif col.color_space.ncomponents == 4:
return f"cmyk({','.join(map(str, col.values))})"
return f"{col.color_space.name}({','.join(map(str, col.values))})"
else:
return default
def render_curve(html: HTMLMaker, item: LTCurve):
css = {
'position': 'absolute',
'left': f'{item.x0}px',
'bottom': f'{item.y0}px',
'width': f'{item.width}px',
'height': f'{item.height}px',
}
path = ""
svg_attr = {
'class': 'curve',
'z-index': 2
}
path_css = {
}
# Compute ViewBox
p_vals = { "x": [], "y": [] }
for p in item.paths:
for pt in p.points:
p_vals['x'].append(pt.x)
p_vals['y'].append(pt.y)
mx_x = max([*p_vals['x'], 1])
mn_x = min([*p_vals['x'], 0])
mx_y = max([*p_vals['y'], 1])
mn_y = min([*p_vals['y'], 0])
if mx_x == mn_x:
mn_x -= 1
if mx_y == mn_y:
mn_y -= 1
dx = 1/( mx_x - mn_x )
dy = (item.height/item.width)/( mx_y - mn_y )
ROUND_VAL = 5
svg_attr.update({
# "viewBox": f"{min(p_vals['x'])} {min(p_vals['y'])} {max(p_vals['x'])} {max(p_vals['y'])}",
"viewBox": f"{round(mn_x*dx, ROUND_VAL)} {round(mn_y*dy, ROUND_VAL)} {round(mx_x*dx , ROUND_VAL)} {round(mx_y*dy, ROUND_VAL)}",
})
# Compute shape
for p in item.paths:
if p.method == CurvePath.METHOD.MOVE_TO:
path += f'M{" ".join( f"{round(dx*pt.x, ROUND_VAL)} {round(dy*pt.y, ROUND_VAL)}" for pt in p.points )} '
if p.method == CurvePath.METHOD.LINE_TO:
path += f'L{" ".join( f"{round(dx*pt.x, ROUND_VAL)} {round(dy*pt.y, ROUND_VAL)}" for pt in p.points )} '
if p.method == CurvePath.METHOD.CLOSE_PATH:
path += 'Z'
stroke_col = get_color(item.stroke)
fill_col = get_color(item.fill)
if fill_col:
path_css['fill'] = fill_col
if stroke_col:
path_css['stroke'] = stroke_col
with html.elm('svg', svg_attr, css):
html.singleton('path', {
"d": path,
**path_css
}, {
**path_css
})
+134
View File
@@ -0,0 +1,134 @@
from typing import List
from .writers.json import JSONMaker, JSONMakerObject, JSONMakerArray
from ..interpreter import PDFInterpreter, PageInterpreter, logging
from ..interpreter import LTImage, LTCharBlock, LTChar, LTCurve, LTXObject
from ..interpreter.commands import LTItem
from ..interpreter.commands import LTRect, LTLine
from ..interpreter.commands.state import CurvePath, PDFColor
from ..parser.PSStackParser import PSLiteral
from ..utils import enc
def normalize_color_values(values: list):
ret = []
for v in values:
if isinstance(v, PSLiteral):
ret.append( enc(str(v.name)).decode() )
else:
ret.append( enc(str(v)).decode() )
return ret
def convert_to_json(
input_file_path: str,
output_file_path: str,
image_folder_path: str = None,
dont_export_images: bool = False,
codec: str = 'utf-8',
maxpages: int = 0,
password: str = None,
caching: bool = True,
check_extractable: bool = True,
pagenos: List[int] = None,
debug_level: int = logging.WARNING,
):
intepreter = PDFInterpreter(input_file_path,
maxpages=maxpages,
password=password,
caching=caching,
check_extractable=check_extractable,
pagenos=pagenos,
debug_level=debug_level
)
with JSONMaker(output_file_path, codec=codec) as json:
with json.array("pages") as arr:
for page in intepreter:
with arr.object() as page_obj:
render_page(page_obj, page)
def render_page(json: JSONMakerObject, ltpage: PageInterpreter):
json.number("id", ltpage.page_num)
json.number("width", ltpage.width)
json.number("height", ltpage.height)
with json.array("children") as chil_arr:
for child in ltpage:
with chil_arr.object() as chil_obj:
render(chil_obj, child)
def render(json: JSONMakerObject, item: LTItem):
if isinstance(item, LTCurve):
place_curve(json, item)
elif isinstance(item, LTXObject):
json.string("type", 'xobject')
json.number("x0", item.x0)
json.number("x1", item.x1)
json.number("y0", item.y0)
json.number("y1", item.y1)
with json.array("children") as chil_arr:
for child in item:
with chil_arr.object() as chil_obj:
render(chil_obj, child)
elif isinstance(item, LTImage):
# if json.imagewriter is not None:
# name = json.imagewriter.export_image(item)
json.string("type", 'img')
json.string("src", enc(item.name, None))
json.number("x0", item.x0)
json.number("x1", item.x1)
json.number("y0", item.y0)
json.number("y1", item.y1)
elif isinstance(item, LTCharBlock):
place_char_block(json, item)
def place_char_block(json: JSONMakerObject, char_block: LTCharBlock):
json.string("type", 'char-block')
json.number("size", char_block.size)
json.number("x0", char_block.x0)
json.number("x1", char_block.x1)
json.number("y0", char_block.y0)
json.number("y1", char_block.y1)
if char_block.color.color_space is not None:
json.place_object("color", {
"type": enc(char_block.color.color_space.name).decode(),
"value": normalize_color_values(char_block.color.values)
})
for key, value in char_block.font.descriptor.items():
if key != "Type" and "FontFile" not in key:
json.string(key, value)
json.string("text",
"".join([
char.get_text()
for char in char_block
])
)
def place_curve(json: JSONMakerObject, item: LTCurve):
json.number("x0", item.x0)
json.number("x1", item.x1)
json.number("y0", item.y0)
json.number("y1", item.y1)
if item.fill.color_space is not None:
json.place_object("fill", {
"type": enc(item.fill.color_space.name).decode(),
"value": normalize_color_values(item.fill.values)
})
if item.stroke.color_space is not None:
json.place_object("stroke", {
"type": enc(item.stroke.color_space.name).decode(),
"value": normalize_color_values(item.stroke.values)
})
if isinstance(item, LTRect):
json.string("type", 'rect')
elif isinstance(item, LTLine):
json.string("type", 'line')
else:
json.string("type", 'curve')
with json.array('paths') as paths_arr:
for path in item.paths:
paths_arr.place_object({
"type": path.method.name,
"points": [
{"x": point.x, "y": point.y}
for point in path.points
]
})
+39
View File
@@ -0,0 +1,39 @@
from typing import List
from ..interpreter import PDFInterpreter, PageInterpreter, logging
from ..interpreter import LTCharBlock, LTXObject
def convert_to_text(
input_file_path: str,
output_file_path: str,
image_folder_path: str = None,
dont_export_images: bool = False,
codec: str = 'utf-8',
maxpages: int = 0,
password: str = None,
caching: bool = True,
check_extractable: bool = True,
pagenos: List[int] = None,
debug_level: int = logging.WARNING,
):
intepreter = PDFInterpreter(input_file_path,
maxpages=maxpages,
password=password,
caching=caching,
check_extractable=check_extractable,
pagenos=pagenos,
debug_level=debug_level
)
with open(output_file_path, 'wb') as outfp:
def process_container(container: LTXObject):
for item in container:
if isinstance(item, LTCharBlock):
for char in item:
outfp.write(char.get_text().encode(codec))
elif isinstance(item, LTXObject):
process_container(item)
for page in intepreter:
outfp.write(f"========== [ page {page.page_num} ] ==========\n".encode(codec))
process_container(page)
outfp.write('\n'.encode(codec))
+62
View File
@@ -0,0 +1,62 @@
from io import TextIOWrapper
from contextlib import contextmanager
class HTMLMaker:
class AccessError(Exception): pass
def __init__(self, file_path: str, codec: str = 'utf-8'):
self.file_path = file_path
self.codec = codec
self.outfile: TextIOWrapper = None
def __enter__(self):
self.outfile = open(self.file_path, 'wb')
self.__levels_deep = 0
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.outfile.close()
self.outfile = None
def write_raw(self, text):
if self.outfile is not None:
if self.codec:
text = text.encode(self.codec)
self.outfile.write(text)
else:
raise self.AccessError("Attempting to write to file without an open connection")
def write(self, text: str, lineend: str = '\n', deep_space: str = ' '):
text = deep_space*self.__levels_deep + text + lineend
self.write_raw(text)
@contextmanager
def elm(self, tag_name: str, attrs: dict = None, css: dict = None, nolineend: bool = False, no_additional_char: bool = False, singleton: bool = False):
lineend = '\n'
deep_space = ' '
if no_additional_char:
lineend = ''
deep_space = ''
if attrs is None:
attrs = {}
if css is not None:
attrs['style'] = ";".join([
f'{name}: {value}'
for name, value in css.items()
])
attrs = " ".join([
f'{name}="{value}"'
for name, value in attrs.items()
])
if not singleton:
self.write(f"<{tag_name} {attrs}>", lineend=lineend)
self.__levels_deep += 1
yield
self.__levels_deep -= 1
self.write(f"</{tag_name}>", deep_space=deep_space, lineend='\n' if not nolineend else '')
else:
self.write(f"<{tag_name} {attrs} />", lineend=lineend, deep_space=deep_space)
yield
def singleton(self, tag_name: str, attrs: dict = None, css: dict = None):
with self.elm(tag_name, attrs, css, singleton=True):
pass
+129
View File
@@ -0,0 +1,129 @@
from json import dumps as json_dump
from io import TextIOWrapper
from contextlib import contextmanager
@contextmanager
def JSONMaker(file_path: str, codec: str = 'utf-8'):
with open(file_path, 'wb') as outf:
with JSONMakerObject(outf, codec=codec) as obj:
yield obj
class JSONMakerObject:
class AccessError(Exception): pass
def __init__(self, outfile: TextIOWrapper, codec: str = 'utf-8', levels_deep: int = 0):
self.outfile: TextIOWrapper = outfile
self.codec = codec
self.levels_deep = levels_deep
self.keys = set()
def __enter__(self):
self.write_raw('{')
self.levels_deep += 1
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.levels_deep -= 1
self.write('}', prefix='\n')
def write_raw(self, text):
if self.outfile is not None:
if self.codec:
text = text.encode(self.codec)
self.outfile.write(text)
else:
raise self.AccessError("Attempting to write to file without an open connection")
def write(self, text: str, deep_space: str = ' ', prefix:str = ''):
text = prefix + (deep_space*self.levels_deep) + text
self.write_raw(text)
def write_prop(self, key: str, val: str = ""):
if key not in self.keys:
self.keys.add(key)
if len(self.keys) > 1:
self.write(f'"{key}": {val}', prefix=',\n')
else:
self.write(f'"{key}": {val}', prefix='\n')
def number(self, key: str, val: float or int):
self.write_prop(key, val)
def string(self, key: str, val: str):
self.write_prop(key, f'"{val}"')
@contextmanager
def object(self, key: str):
self.write_prop(key)
with JSONMakerObject(self.outfile, codec=self.codec, levels_deep=self.levels_deep) as child:
yield child
@contextmanager
def array(self, key: str):
self.write_prop(key)
with JSONMakerArray(self.outfile, codec=self.codec, levels_deep=self.levels_deep) as child:
yield child
def place_object(self, key: str, obj: dict):
self.write_prop(key, json_dump(obj))
class JSONMakerArray:
class AccessError(Exception): pass
def __init__(self, outfile: TextIOWrapper, codec: str = 'utf-8', levels_deep: int = 0):
self.outfile: TextIOWrapper = outfile
self.codec = codec
self.levels_deep = levels_deep
self.childrencount = 0
def __enter__(self):
self.write_raw('[')
self.levels_deep += 1
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.levels_deep -= 1
self.write(']', prefix='\n')
def write_raw(self, text):
if self.outfile is not None:
if self.codec:
text = text.encode(self.codec)
self.outfile.write(text)
else:
raise self.AccessError("Attempting to write to file without an open connection")
def write(self, text: str, deep_space: str = ' ', prefix:str = ''):
text = prefix + (deep_space*self.levels_deep) + text
self.write_raw(text)
def write_prop(self, val: str = ""):
self.childrencount += 1
if self.childrencount > 1:
self.write(f'{val}', prefix=',\n')
else:
self.write(f'{val}', prefix='\n')
def number(self, val: float or int):
self.write_prop(val)
def string(self, val: str):
self.write_prop(f'"{val}"')
@contextmanager
def object(self):
self.write_prop()
with JSONMakerObject(self.outfile, codec=self.codec, levels_deep=self.levels_deep) as child:
yield child
@contextmanager
def array(self):
self.write_prop()
with JSONMakerArray(self.outfile, codec=self.codec, levels_deep=self.levels_deep) as child:
yield child
def place_object(self, obj: dict):
self.write_prop(json_dump(obj))
+60
View File
@@ -0,0 +1,60 @@
from io import TextIOWrapper
from contextlib import contextmanager
class XMLMaker:
class AccessError(Exception): pass
def __init__(self, file_path: str, codec: str = 'utf-8'):
self.file_path = file_path
self.codec = codec
self.outfile: TextIOWrapper = None
def __enter__(self):
self.outfile = open(self.file_path, 'wb')
if self.codec:
self.write_raw(f'<?xml version="1.0.0" encoding="{self.codec}" ?>')
else:
self.write_raw(f'<?xml version="1.0.0" ?>')
self.__levels_deep = 0
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.outfile.close()
self.outfile = None
def write_raw(self, text):
if self.outfile is not None:
if self.codec:
text = text.encode(self.codec)
self.outfile.write(text)
else:
raise self.AccessError("Attempting to write to file without an open connection")
def write(self, text: str, lineend: str = '\n', deep_space: str = ' '):
text = deep_space*self.__levels_deep + text + lineend
self.write_raw(text)
@contextmanager
def elm(self, tag_name: str, attrs: dict = None, nolineend: bool = False, no_additional_char: bool = False, singleton: bool = False):
lineend = '\n'
deep_space = ' '
if no_additional_char:
lineend = ''
deep_space = ''
if attrs is None:
attrs = {}
attrs = " ".join([
f'{name}="{value}"'
for name, value in attrs.items()
])
if not singleton:
self.write(f"<{tag_name} {attrs}>", lineend=lineend)
self.__levels_deep += 1
yield
self.__levels_deep -= 1
self.write(f"</{tag_name}>", deep_space=deep_space, lineend='\n' if not nolineend else '')
else:
self.write(f"<{tag_name} {attrs} />", lineend=lineend, deep_space=deep_space)
yield
def singleton(self, tag_name: str, attrs: dict = None):
with self.elm(tag_name, attrs, singleton=True):
pass
+142
View File
@@ -0,0 +1,142 @@
import re
from typing import List
from xml.sax import saxutils
from .writers.xml import XMLMaker
from ..interpreter import PDFInterpreter, PageInterpreter, logging
from ..interpreter import LTImage, LTCharBlock, LTChar, LTCurve, LTXObject
from ..interpreter.commands import LTItem
from ..interpreter.commands import LTRect, LTLine
from ..interpreter.commands.state import CurvePath, PDFColor
from ..parser.PSStackParser import PSLiteral
from ..utils import enc
FIRST_CAP_RE = re.compile('(.)([A-Z][a-z]+)')
ALL_CAP_RE = re.compile('([a-z0-9])([A-Z])')
def convert_capcase_to_snakecase(text: str) -> str:
s1 = FIRST_CAP_RE.sub(r'\1-\2', text)
return ALL_CAP_RE.sub(r'\1-\2', s1).lower()
def normalize_color_values(values: list):
ret = ""
for i,v in enumerate(values):
if i > 0:
ret += ','
if isinstance(v, PSLiteral):
ret += enc(str(v.name)).decode()
else:
ret += enc(str(v)).decode()
return ret
def get_color(col: PDFColor, prefix: str = 'color', codec = 'utf-8'):
if col.color_space is not None:
return {
f"{prefix}-type": enc(col.color_space.name).decode(),
f"{prefix}-values": normalize_color_values(col.values)
}
return {}
def convert_to_xml(
input_file_path: str,
output_file_path: str,
image_folder_path: str = None,
dont_export_images: bool = False,
codec: str = 'utf-8',
maxpages: int = 0,
password: str = None,
caching: bool = True,
check_extractable: bool = True,
pagenos: List[int] = None,
debug_level: int = logging.WARNING,
):
intepreter = PDFInterpreter(input_file_path,
maxpages=maxpages,
password=password,
caching=caching,
check_extractable=check_extractable,
pagenos=pagenos,
debug_level=debug_level
)
with XMLMaker(output_file_path, codec=codec) as xml:
with xml.elm("pages"):
for page in intepreter:
render_page(xml, page)
def render_page(xml: XMLMaker, ltpage: PageInterpreter):
def render(item: LTItem):
if isinstance(item, LTCurve):
place_curve(xml, item)
elif isinstance(item, LTXObject):
with xml.elm('xobject', {
"x0": item.x0,
"x1": item.x1,
"y0": item.y0,
"y1": item.y1,
}):
for child in item:
render(child)
elif isinstance(item, LTImage):
name = item.name
# if xml.imagewriter is not None:
# name = xml.imagewriter.export_image(item)
xml.singleton('img', attrs={
'src': enc(name, None),
"x0": item.x0,
"x1": item.x1,
"y0": item.y0,
"y1": item.y1,
})
elif isinstance(item, LTCharBlock):
place_char_block(xml, item)
with xml.elm('page', {
"id": ltpage.page_num,
"width": ltpage.width,
"height": ltpage.height,
}):
for child in ltpage:
render(child)
def place_char(xml: XMLMaker, char: LTChar):
with xml.elm('char', {
'x0': char.x0,
'x1': char.x1,
'y0': char.y0,
'y1': char.y1,
}, no_additional_char=True):
xml.write(saxutils.escape(char.get_text()), lineend='', deep_space="")
def place_char_block(xml: XMLMaker, char_block: LTCharBlock):
attr = {
'size': char_block.size,
"x0": char_block.x0,
"x1": char_block.x1,
"y0": char_block.y0,
"y1": char_block.y1,
**get_color(char_block.color, codec=xml.codec),
}
for key, value in char_block.font.descriptor.items():
if key != "Type" and "FontFile" not in key:
attr[convert_capcase_to_snakecase(key)] = value
with xml.elm("char-block", attr):
for char in char_block:
place_char(xml, char)
def place_curve(xml: XMLMaker, item: LTCurve):
attr = {
"x0": item.x0,
"x1": item.x1,
"y0": item.y0,
"y1": item.y1,
**get_color(item.stroke, "stroke", codec=xml.codec),
**get_color(item.fill, "fill", codec=xml.codec),
}
if isinstance(item, LTRect):
return xml.singleton('rect', attr)
elif isinstance(item, LTLine):
return xml.singleton('line', attr)
with xml.elm('curve', attr):
for path in item.paths:
with xml.elm('path', { "type": path.method.name }):
for point in path.points:
xml.singleton('point', { 'x': point.x, 'y': point.y })
-12
View File
@@ -1,12 +0,0 @@
from ..utils import get_logger
log = get_logger(__name__)
from ..layouts import PDFLayoutAnalyzer, PDFPage, LTPage
class PDFExctractor(PDFLayoutAnalyzer):
def end_page(self, page: PDFPage):
assert not self._stack, str(len(self._stack))
assert isinstance(self.cur_item, LTPage), str(type(self.cur_item))
self.pageno += 1
return self.cur_item
-46
View File
@@ -1,46 +0,0 @@
import re
import os
from typing import List
from io import BytesIO, StringIO, TextIOWrapper
from tqdm import tqdm
from ..interpreter.PDFResourceManager import PDFResourceManager
from ..interpreter.PDFPage import PDFPage
from ..interpreter.PDFPageInterpreter import PDFPageInterpreter
from ..imagewriter import ImageWriter
from ..utils import set_log_level, get_logger, logging
from .PDFExtractor import PDFExctractor
log = get_logger(__name__)
def extract_items_from_pdf(
input_file_path: str,
maxpages: int = 0,
password: str = None,
caching: bool = True,
check_extractable: bool = True,
pagenos: List[int] = None,
debug_level: int = logging.WARNING,
):
set_log_level(debug_level)
log.debug("Creating resources....")
rsrcmgr = PDFResourceManager(caching=caching)
device = PDFExctractor(
rsrcmgr=rsrcmgr,
pageno=0
)
interpreter = PDFPageInterpreter(rsrcmgr, device)
log.debug("Reading pages....")
with open(input_file_path, 'rb') as input_file:
pages = PDFPage.get_pages(input_file, pagenos=pagenos, maxpages=maxpages, password=password, caching=caching, check_extractable=check_extractable)
page_waiter = tqdm(enumerate(pages), disable=debug_level > logging.INFO)
for _, page in page_waiter:
page_waiter.set_description("Processing pdf...")
yield interpreter.process_page(page)
page_waiter.close()
device.close()
-755
View File
@@ -1,755 +0,0 @@
import sys
import struct
from io import BytesIO
from abc import abstractmethod
from .cmapdb import CMapDB
from .cmapdb import CMapParser
from .cmapdb import FileUnicodeMap
from .cmapdb import CMap
from .EncodingDB import EncodingDB, name2unicode
from .PSStackParser import PSStackParser
from .PSStackParser import PSEOF
from .PSStackParser import LIT
from .PSStackParser import KWD
from .PSStackParser import PSLiteral
from .PSStackParser import literal_name
from .PDFStream import PDFException
from .PDFStream import resolve1
from .PDFStream import int_value
from .PDFStream import num_value
from .PDFStream import list_value
from .PDFStream import dict_value
from .PDFStream import PDFStream
from ..utils import settings, FONT_METRICS, apply_matrix_norm, nunpack, choplist, isnumber
from .constants import COMMON_FONT_ATTRIBUTES
def get_widths(seq):
widths = {}
r = []
for v in seq:
if isinstance(v, list):
if r:
char1 = r[-1]
for (i, w) in enumerate(v):
widths[char1+i] = w
r = []
elif isnumber(v):
r.append(v)
if len(r) == 3:
(char1, char2, w) = r
for i in range(char1, char2+1):
widths[i] = w
r = []
return widths
#assert get_widths([1]) == {}
#assert get_widths([1,2,3]) == {1:3, 2:3}
#assert get_widths([1,[2,3],6,[7,8]]) == {1:2,2:3, 6:7,7:8}
def get_widths2(seq):
widths = {}
r = []
for v in seq:
if isinstance(v, list):
if r:
char1 = r[-1]
for (i, (w, vx, vy)) in enumerate(choplist(3, v)):
widths[char1+i] = (w, (vx, vy))
r = []
elif isnumber(v):
r.append(v)
if len(r) == 5:
(char1, char2, w, vx, vy) = r # pylint: disable=E0632
for i in range(char1, char2+1):
widths[i] = (w, (vx, vy))
r = []
return widths
#assert get_widths2([1]) == {}
#assert get_widths2([1,2,3,4,5]) == {1:(3, (4,5)), 2:(3, (4,5))}
#assert get_widths2([1,[2,3,4,5],6,[7,8,9]]) == {1:(2, (3,4)), 6:(7, (8,9))}
## FontMetricsDB
##
class FontMetricsDB(object):
@classmethod
def get_metrics(klass, fontname):
return FONT_METRICS[fontname]
## Type1FontHeaderParser
##
class Type1FontHeaderParser(PSStackParser):
KEYWORD_BEGIN = KWD(b'begin')
KEYWORD_END = KWD(b'end')
KEYWORD_DEF = KWD(b'def')
KEYWORD_PUT = KWD(b'put')
KEYWORD_DICT = KWD(b'dict')
KEYWORD_ARRAY = KWD(b'array')
KEYWORD_READONLY = KWD(b'readonly')
KEYWORD_FOR = KWD(b'for')
KEYWORD_FOR = KWD(b'for')
def __init__(self, data):
PSStackParser.__init__(self, data)
self._cid2unicode = {}
return
def get_encoding(self):
while 1:
try:
(cid, name) = self.nextobject()
except PSEOF:
break
try:
self._cid2unicode[cid] = name2unicode(name)
except KeyError:
pass
return self._cid2unicode
def do_keyword(self, pos, token):
if token is self.KEYWORD_PUT:
((_, key), (_, value)) = self.pop(2) # pylint: disable=E0632
if (isinstance(key, int) and
isinstance(value, PSLiteral)):
self.add_results((key, literal_name(value)))
return
NIBBLES = ('0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '.', 'e', 'e-', None, '-')
## CFFFont
## (Format specified in Adobe Technical Note: #5176
## "The Compact Font Format Specification")
##
def getdict(data):
d = {}
fp = BytesIO(data)
stack = []
while 1:
c = fp.read(1)
if not c:
break
b0 = ord(c)
if b0 <= 21:
d[b0] = stack
stack = []
continue
if b0 == 30:
s = ''
loop = True
while loop:
b = ord(fp.read(1))
for n in (b >> 4, b & 15):
if n == 15:
loop = False
else:
s += NIBBLES[n]
value = float(s)
elif 32 <= b0 and b0 <= 246:
value = b0-139
else:
b1 = ord(fp.read(1))
if 247 <= b0 and b0 <= 250:
value = ((b0-247) << 8)+b1+108
elif 251 <= b0 and b0 <= 254:
value = -((b0-251) << 8)-b1-108
else:
b2 = ord(fp.read(1))
if 128 <= b1:
b1 -= 256
if b0 == 28:
value = b1 << 8 | b2
else:
value = b1 << 24 | b2 << 16 | struct.unpack('>H', fp.read(2))[0]
stack.append(value)
return d
class CFFFont(object):
STANDARD_STRINGS = (
'.notdef', 'space', 'exclam', 'quotedbl', 'numbersign',
'dollar', 'percent', 'ampersand', 'quoteright', 'parenleft',
'parenright', 'asterisk', 'plus', 'comma', 'hyphen', 'period',
'slash', 'zero', 'one', 'two', 'three', 'four', 'five', 'six',
'seven', 'eight', 'nine', 'colon', 'semicolon', 'less', 'equal',
'greater', 'question', 'at', 'A', 'B', 'C', 'D', 'E', 'F', 'G',
'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T',
'U', 'V', 'W', 'X', 'Y', 'Z', 'bracketleft', 'backslash',
'bracketright', 'asciicircum', 'underscore', 'quoteleft', 'a',
'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n',
'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z',
'braceleft', 'bar', 'braceright', 'asciitilde', 'exclamdown',
'cent', 'sterling', 'fraction', 'yen', 'florin', 'section',
'currency', 'quotesingle', 'quotedblleft', 'guillemotleft',
'guilsinglleft', 'guilsinglright', 'fi', 'fl', 'endash',
'dagger', 'daggerdbl', 'periodcentered', 'paragraph', 'bullet',
'quotesinglbase', 'quotedblbase', 'quotedblright',
'guillemotright', 'ellipsis', 'perthousand', 'questiondown',
'grave', 'acute', 'circumflex', 'tilde', 'macron', 'breve',
'dotaccent', 'dieresis', 'ring', 'cedilla', 'hungarumlaut',
'ogonek', 'caron', 'emdash', 'AE', 'ordfeminine', 'Lslash',
'Oslash', 'OE', 'ordmasculine', 'ae', 'dotlessi', 'lslash',
'oslash', 'oe', 'germandbls', 'onesuperior', 'logicalnot', 'mu',
'trademark', 'Eth', 'onehalf', 'plusminus', 'Thorn',
'onequarter', 'divide', 'brokenbar', 'degree', 'thorn',
'threequarters', 'twosuperior', 'registered', 'minus', 'eth',
'multiply', 'threesuperior', 'copyright', 'Aacute',
'Acircumflex', 'Adieresis', 'Agrave', 'Aring', 'Atilde',
'Ccedilla', 'Eacute', 'Ecircumflex', 'Edieresis', 'Egrave',
'Iacute', 'Icircumflex', 'Idieresis', 'Igrave', 'Ntilde',
'Oacute', 'Ocircumflex', 'Odieresis', 'Ograve', 'Otilde',
'Scaron', 'Uacute', 'Ucircumflex', 'Udieresis', 'Ugrave',
'Yacute', 'Ydieresis', 'Zcaron', 'aacute', 'acircumflex',
'adieresis', 'agrave', 'aring', 'atilde', 'ccedilla', 'eacute',
'ecircumflex', 'edieresis', 'egrave', 'iacute', 'icircumflex',
'idieresis', 'igrave', 'ntilde', 'oacute', 'ocircumflex',
'odieresis', 'ograve', 'otilde', 'scaron', 'uacute',
'ucircumflex', 'udieresis', 'ugrave', 'yacute', 'ydieresis',
'zcaron', 'exclamsmall', 'Hungarumlautsmall', 'dollaroldstyle',
'dollarsuperior', 'ampersandsmall', 'Acutesmall',
'parenleftsuperior', 'parenrightsuperior', 'twodotenleader',
'onedotenleader', 'zerooldstyle', 'oneoldstyle', 'twooldstyle',
'threeoldstyle', 'fouroldstyle', 'fiveoldstyle', 'sixoldstyle',
'sevenoldstyle', 'eightoldstyle', 'nineoldstyle',
'commasuperior', 'threequartersemdash', 'periodsuperior',
'questionsmall', 'asuperior', 'bsuperior', 'centsuperior',
'dsuperior', 'esuperior', 'isuperior', 'lsuperior', 'msuperior',
'nsuperior', 'osuperior', 'rsuperior', 'ssuperior', 'tsuperior',
'ff', 'ffi', 'ffl', 'parenleftinferior', 'parenrightinferior',
'Circumflexsmall', 'hyphensuperior', 'Gravesmall', 'Asmall',
'Bsmall', 'Csmall', 'Dsmall', 'Esmall', 'Fsmall', 'Gsmall',
'Hsmall', 'Ismall', 'Jsmall', 'Ksmall', 'Lsmall', 'Msmall',
'Nsmall', 'Osmall', 'Psmall', 'Qsmall', 'Rsmall', 'Ssmall',
'Tsmall', 'Usmall', 'Vsmall', 'Wsmall', 'Xsmall', 'Ysmall',
'Zsmall', 'colonmonetary', 'onefitted', 'rupiah', 'Tildesmall',
'exclamdownsmall', 'centoldstyle', 'Lslashsmall', 'Scaronsmall',
'Zcaronsmall', 'Dieresissmall', 'Brevesmall', 'Caronsmall',
'Dotaccentsmall', 'Macronsmall', 'figuredash', 'hypheninferior',
'Ogoneksmall', 'Ringsmall', 'Cedillasmall', 'questiondownsmall',
'oneeighth', 'threeeighths', 'fiveeighths', 'seveneighths',
'onethird', 'twothirds', 'zerosuperior', 'foursuperior',
'fivesuperior', 'sixsuperior', 'sevensuperior', 'eightsuperior',
'ninesuperior', 'zeroinferior', 'oneinferior', 'twoinferior',
'threeinferior', 'fourinferior', 'fiveinferior', 'sixinferior',
'seveninferior', 'eightinferior', 'nineinferior',
'centinferior', 'dollarinferior', 'periodinferior',
'commainferior', 'Agravesmall', 'Aacutesmall',
'Acircumflexsmall', 'Atildesmall', 'Adieresissmall',
'Aringsmall', 'AEsmall', 'Ccedillasmall', 'Egravesmall',
'Eacutesmall', 'Ecircumflexsmall', 'Edieresissmall',
'Igravesmall', 'Iacutesmall', 'Icircumflexsmall',
'Idieresissmall', 'Ethsmall', 'Ntildesmall', 'Ogravesmall',
'Oacutesmall', 'Ocircumflexsmall', 'Otildesmall',
'Odieresissmall', 'OEsmall', 'Oslashsmall', 'Ugravesmall',
'Uacutesmall', 'Ucircumflexsmall', 'Udieresissmall',
'Yacutesmall', 'Thornsmall', 'Ydieresissmall', '001.000',
'001.001', '001.002', '001.003', 'Black', 'Bold', 'Book',
'Light', 'Medium', 'Regular', 'Roman', 'Semibold',
)
class INDEX(object):
def __init__(self, fp):
self.fp = fp
self.offsets = []
(count, offsize) = struct.unpack('>HB', self.fp.read(3))
for _ in range(count+1):
self.offsets.append(nunpack(self.fp.read(offsize)))
self.base = self.fp.tell()-1
self.fp.seek(self.base+self.offsets[-1])
return
def __repr__(self):
return '<INDEX: size=%d>' % len(self)
def __len__(self): # pylint: disable=E0303
return len(self.offsets)-1
def __getitem__(self, i):
self.fp.seek(self.base+self.offsets[i])
return self.fp.read(self.offsets[i+1]-self.offsets[i])
def __iter__(self):
return iter(self[i] for i in range(len(self)))
def __init__(self, name, fp):
self.name = name
self.fp = fp
# Header
(_major, _minor, hdrsize, _) = struct.unpack('BBBB', self.fp.read(4))
self.fp.read(hdrsize-4)
# Name INDEX
self.name_index = self.INDEX(self.fp)
# Top DICT INDEX
self.dict_index = self.INDEX(self.fp)
# String INDEX
self.string_index = self.INDEX(self.fp)
# Global Subr INDEX
self.subr_index = self.INDEX(self.fp)
# Top DICT DATA
self.top_dict = getdict(self.dict_index[0])
(charset_pos,) = self.top_dict.get(15, [0])
(encoding_pos,) = self.top_dict.get(16, [0])
(charstring_pos,) = self.top_dict.get(17, [0])
# CharStrings
self.fp.seek(charstring_pos)
self.charstring = self.INDEX(self.fp)
self.nglyphs = len(self.charstring)
# Encodings
self.code2gid = {}
self.gid2code = {}
self.fp.seek(encoding_pos)
format = self.fp.read(1)
if format == b'\x00':
# Format 0
(n,) = struct.unpack('B', self.fp.read(1))
for (code, gid) in enumerate(struct.unpack('B'*n, self.fp.read(n))):
self.code2gid[code] = gid
self.gid2code[gid] = code
elif format == b'\x01':
# Format 1
(n,) = struct.unpack('B', self.fp.read(1))
code = 0
for _ in range(n):
(first, nleft) = struct.unpack('BB', self.fp.read(2))
for gid in range(first, first+nleft+1):
self.code2gid[code] = gid
self.gid2code[gid] = code
code += 1
else:
raise ValueError('unsupported encoding format: %r' % format)
# Charsets
self.name2gid = {}
self.gid2name = {}
self.fp.seek(charset_pos)
format = self.fp.read(1)
if format == b'\x00':
# Format 0
n = self.nglyphs-1
for (gid, sid) in enumerate(struct.unpack('>'+'H'*n, self.fp.read(2*n))):
gid += 1
name = self.getstr(sid)
self.name2gid[name] = gid
self.gid2name[gid] = name
elif format == b'\x01':
# Format 1
(n,) = struct.unpack('B', self.fp.read(1))
sid = 0
for _ in range(n):
(first, nleft) = struct.unpack('BB', self.fp.read(2))
for gid in range(first, first+nleft+1):
name = self.getstr(sid)
self.name2gid[name] = gid
self.gid2name[gid] = name
sid += 1
elif format == b'\x02':
# Format 2
assert False, str(('Unhandled', format))
else:
raise ValueError('unsupported charset format: %r' % format)
#print self.code2gid
#print self.name2gid
#assert 0
return
def getstr(self, sid):
if sid < len(self.STANDARD_STRINGS):
return self.STANDARD_STRINGS[sid]
return self.string_index[sid-len(self.STANDARD_STRINGS)]
## TrueTypeFont
##
class TrueTypeFont(object):
class CMapNotFound(Exception):
pass
def __init__(self, name, fp):
self.name = name
self.fp = fp
self.tables = {}
self.fonttype = fp.read(4)
try:
(ntables, _1, _2, _3) = struct.unpack('>HHHH', fp.read(8))
for _ in range(ntables):
(name, _, offset, length) = struct.unpack('>4sLLL', fp.read(16))
self.tables[name] = (offset, length)
except struct.error:
# Do not fail if there are not enough bytes to read. Even for
# corrupted PDFs we would like to get as much information as
# possible, so continue.
pass
return
def create_unicode_map(self):
if 'cmap' not in self.tables:
raise TrueTypeFont.CMapNotFound
(base_offset, _) = self.tables['cmap']
fp = self.fp
fp.seek(base_offset)
(_, nsubtables) = struct.unpack('>HH', fp.read(4))
subtables = []
for i in range(nsubtables):
subtables.append(struct.unpack('>HHL', fp.read(8)))
char2gid = {}
# Only supports subtable type 0, 2 and 4.
for (_1, _2, st_offset) in subtables:
fp.seek(base_offset+st_offset)
(fmttype, _, _) = struct.unpack('>HHH', fp.read(6))
if fmttype == 0:
char2gid.update(enumerate(struct.unpack('>256B', fp.read(256))))
elif fmttype == 2:
subheaderkeys = struct.unpack('>256H', fp.read(512))
firstbytes = [0]*8192
for (i, k) in enumerate(subheaderkeys):
firstbytes[k//8] = i
nhdrs = max(subheaderkeys)//8 + 1
hdrs = []
for i in range(nhdrs):
(firstcode, entcount, delta, offset) = struct.unpack('>HHhH', fp.read(8))
hdrs.append((i, firstcode, entcount, delta, fp.tell()-2+offset))
for (i, firstcode, entcount, delta, pos) in hdrs:
if not entcount:
continue
first = firstcode + (firstbytes[i] << 8)
fp.seek(pos)
for c in range(entcount):
gid = struct.unpack('>H', fp.read(2))
if gid:
gid += delta
char2gid[first+c] = gid
elif fmttype == 4:
(segcount, _1, _2, _3) = struct.unpack('>HHHH', fp.read(8))
segcount //= 2
ecs = struct.unpack('>%dH' % segcount, fp.read(2*segcount))
fp.read(2)
scs = struct.unpack('>%dH' % segcount, fp.read(2*segcount))
idds = struct.unpack('>%dh' % segcount, fp.read(2*segcount))
pos = fp.tell()
idrs = struct.unpack('>%dH' % segcount, fp.read(2*segcount))
for (ec, sc, idd, idr) in zip(ecs, scs, idds, idrs):
if idr:
fp.seek(pos+idr)
for c in range(sc, ec+1):
char2gid[c] = (struct.unpack('>H', fp.read(2))[0] + idd) & 0xffff
else:
for c in range(sc, ec+1):
char2gid[c] = (c + idd) & 0xffff
else:
assert False, str(('Unhandled', fmttype))
# create unicode map
unicode_map = FileUnicodeMap()
for (char, gid) in char2gid.items():
unicode_map.add_cid2unichr(gid, char)
return unicode_map
## Fonts
##
class PDFFontError(PDFException):
pass
class PDFUnicodeNotDefined(PDFFontError):
pass
LITERAL_STANDARD_ENCODING = LIT('StandardEncoding')
LITERAL_TYPE1C = LIT('Type1C')
# PDFFont
class PDFFont(object):
def __init__(self, descriptor, widths, default_width=None):
self.descriptor = descriptor
self.widths = widths
self.fontname = resolve1(descriptor.get('FontName', 'unknown'))
if isinstance(self.fontname, PSLiteral):
self.fontname = literal_name(self.fontname)
self.flags = int_value(descriptor.get('Flags', 0))
self.ascent = num_value(descriptor.get('Ascent', 0))
self.descent = num_value(descriptor.get('Descent', 0))
self.italic_angle = num_value(descriptor.get('ItalicAngle', 0))
self.default_width = default_width or num_value(descriptor.get('MissingWidth', 0))
self.leading = num_value(descriptor.get('Leading', 0))
self.bbox = list_value(descriptor.get('FontBBox', (0, 0, 0, 0)))
self.hscale = self.vscale = .001
return
def __repr__(self):
return '<PDFFont>'
def is_vertical(self):
return False
def is_multibyte(self):
return False
def decode(self, bytes):
return bytearray(bytes) # map(ord, bytes)
def get_ascent(self):
return self.ascent * self.vscale
def get_descent(self):
return self.descent * self.vscale
def get_width(self):
w = self.bbox[2]-self.bbox[0]
if w == 0:
w = -self.default_width
return w * self.hscale
def get_height(self):
h = self.bbox[3]-self.bbox[1]
if h == 0:
h = self.ascent - self.descent
return h * self.vscale
def char_width(self, cid):
try:
return self.widths[cid] * self.hscale
except KeyError:
try:
return self.widths[self.to_unichr(cid)] * self.hscale
except (KeyError, PDFUnicodeNotDefined):
return self.default_width * self.hscale
def char_disp(self, cid):
return 0
def string_width(self, s):
return sum(self.char_width(cid) for cid in self.decode(s))
@abstractmethod
def to_unichr(self, cid):
raise NotImplementedError
@property
def font_weight(self):
if 'FontWeight' in self.descriptor.keys():
return self.descriptor['FontWeight']
elif 'bold' in self.fontname.lower():
return 'bold'
else:
return None
@property
def is_bold(self):
return self.font_weight == 'bold' or self.font_weight > 549
# PDFSimpleFont
class PDFSimpleFont(PDFFont):
def __init__(self, descriptor, widths, spec):
# Font encoding is specified either by a name of
# built-in encoding or a dictionary that describes
# the differences.
if 'Encoding' in spec:
encoding = resolve1(spec['Encoding'])
else:
encoding = LITERAL_STANDARD_ENCODING
if isinstance(encoding, dict):
name = literal_name(encoding.get('BaseEncoding', LITERAL_STANDARD_ENCODING))
diff = list_value(encoding.get('Differences', []))
self.cid2unicode = EncodingDB.get_encoding(name, diff)
else:
self.cid2unicode = EncodingDB.get_encoding(literal_name(encoding))
self.unicode_map = None
if 'ToUnicode' in spec:
strm = PDFStream.validated_stream(spec['ToUnicode'])
self.unicode_map = FileUnicodeMap()
CMapParser(self.unicode_map, BytesIO(strm.get_data())).run()
PDFFont.__init__(self, descriptor, widths)
return
def to_unichr(self, cid):
if self.unicode_map:
try:
return self.unicode_map.get_unichr(cid)
except KeyError:
pass
try:
return self.cid2unicode[cid]
except KeyError:
raise PDFUnicodeNotDefined(None, cid)
# PDFType1Font
class PDFType1Font(PDFSimpleFont):
def __init__(self, rsrcmgr, spec):
try:
self.basefont = literal_name(spec['BaseFont'])
except KeyError:
if settings.STRICT:
raise PDFFontError('BaseFont is missing')
self.basefont = 'unknown'
try:
(descriptor, widths) = FontMetricsDB.get_metrics(self.basefont)
except KeyError:
descriptor = dict_value(spec.get('FontDescriptor', {}))
firstchar = int_value(spec.get('FirstChar', 0))
#lastchar = int_value(spec.get('LastChar', 255))
widths = list_value(spec.get('Widths', [0]*256))
widths = dict((i+firstchar, w) for (i, w) in enumerate(widths))
PDFSimpleFont.__init__(self, descriptor, widths, spec)
if 'Encoding' not in spec and 'FontFile' in descriptor:
# try to recover the missing encoding info from the font file.
self.fontfile = PDFStream.validated_stream(descriptor.get('FontFile'))
length1 = int_value(self.fontfile['Length1'])
data = self.fontfile.get_data()[:length1]
parser = Type1FontHeaderParser(BytesIO(data))
self.cid2unicode = parser.get_encoding()
return
def __repr__(self):
return '<PDFType1Font: basefont=%r>' % self.basefont
# PDFTrueTypeFont
class PDFTrueTypeFont(PDFType1Font):
def __repr__(self):
return '<PDFTrueTypeFont: basefont=%r>' % self.basefont
# PDFType3Font
class PDFType3Font(PDFSimpleFont):
def __init__(self, rsrcmgr, spec):
firstchar = int_value(spec.get('FirstChar', 0))
#lastchar = int_value(spec.get('LastChar', 0))
widths = list_value(spec.get('Widths', [0]*256))
widths = dict((i+firstchar, w) for (i, w) in enumerate(widths))
if 'FontDescriptor' in spec:
descriptor = dict_value(spec['FontDescriptor'])
else:
descriptor = {'Ascent': 0, 'Descent': 0,
'FontBBox': spec['FontBBox']}
PDFSimpleFont.__init__(self, descriptor, widths, spec)
self.matrix = tuple(list_value(spec.get('FontMatrix')))
(_, self.descent, _, self.ascent) = self.bbox
(self.hscale, self.vscale) = apply_matrix_norm(self.matrix, (1, 1))
return
def __repr__(self):
return '<PDFType3Font>'
# PDFCIDFont
class PDFCIDFont(PDFFont):
def __init__(self, rsrcmgr, spec, strict=settings.STRICT):
try:
self.basefont = literal_name(spec['BaseFont'])
except KeyError:
if strict:
raise PDFFontError('BaseFont is missing')
self.basefont = 'unknown'
self.cidsysteminfo = dict_value(spec.get('CIDSystemInfo', {}))
self.cidcoding = '%s-%s' % (resolve1(self.cidsysteminfo.get('Registry', b'unknown')).decode("latin1"),
resolve1(self.cidsysteminfo.get('Ordering', b'unknown')).decode("latin1"))
try:
name = literal_name(spec['Encoding'])
except KeyError:
if strict:
raise PDFFontError('Encoding is unspecified')
name = 'unknown'
try:
self.cmap = CMapDB.get_cmap(name)
except CMapDB.CMapNotFound as e:
if strict:
raise PDFFontError(e)
self.cmap = CMap()
try:
descriptor = dict_value(spec['FontDescriptor'])
except KeyError:
if strict:
raise PDFFontError('FontDescriptor is missing')
descriptor = {}
ttf = None
if 'FontFile2' in descriptor:
self.fontfile = PDFStream.validated_stream(descriptor.get('FontFile2'))
ttf = TrueTypeFont(self.basefont,
BytesIO(self.fontfile.get_data()))
self.unicode_map = None
if 'ToUnicode' in spec:
strm = PDFStream.validated_stream(spec['ToUnicode'])
self.unicode_map = FileUnicodeMap()
CMapParser(self.unicode_map, BytesIO(strm.get_data())).run()
elif self.cidcoding in ('Adobe-Identity', 'Adobe-UCS'):
if ttf:
try:
self.unicode_map = ttf.create_unicode_map()
except TrueTypeFont.CMapNotFound:
pass
else:
try:
self.unicode_map = CMapDB.get_unicode_map(self.cidcoding, self.cmap.is_vertical())
except CMapDB.CMapNotFound as e:
pass
self.vertical = self.cmap.is_vertical()
if self.vertical:
# writing mode: vertical
widths = get_widths2(list_value(spec.get('W2', [])))
self.disps = dict((cid, (vx, vy)) for (cid, (_, (vx, vy))) in iter(widths.items()))
(vy, w) = spec.get('DW2', [880, -1000])
self.default_disp = (None, vy)
widths = dict((cid, w) for (cid, (w, _)) in iter(widths.items()))
default_width = w
else:
# writing mode: horizontal
self.disps = {}
self.default_disp = 0
widths = get_widths(list_value(spec.get('W', [])))
default_width = spec.get('DW', 1000)
PDFFont.__init__(self, descriptor, widths, default_width=default_width)
return
def __repr__(self):
return '<PDFCIDFont: basefont=%r, cidcoding=%r>' % (self.basefont, self.cidcoding)
def is_vertical(self):
return self.vertical
def is_multibyte(self):
return True
def decode(self, bytes):
return self.cmap.decode(bytes)
def char_disp(self, cid):
"Returns an integer for horizontal fonts, a tuple for vertical fonts."
return self.disps.get(cid, self.default_disp)
def to_unichr(self, cid):
try:
if not self.unicode_map:
raise KeyError(cid)
return self.unicode_map.get_unichr(cid)
except KeyError:
raise PDFUnicodeNotDefined(self.cidcoding, cid)
# main
def main(argv):
for fname in argv[1:]:
fp = open(fname, 'rb')
#font = TrueTypeFont(fname, fp)
font = CFFFont(fname, fp)
print (font)
fp.close()
return
if __name__ == '__main__':
sys.exit(main(sys.argv))
@@ -1,75 +0,0 @@
class PDFGraphicStateColor:
class NotSet(Exception): pass
def __init__(self):
self.__rgb = None
self.__gray = None
self.__cmyk = None
self.__custom = None
def get_color(self):
if self.__rgb is not None:
return 'rgb', self.rgb
elif self.__cmyk is not None:
return 'cmyk', self.cmyk
elif self.__gray is not None:
return 'gray', self.gray
elif self.__custom is not None:
return 'custom', self.custom
else: return None, None
@property
def is_set(self):
return self.__rgb is not None or self.__cmyk is not None or self.__gray is not None
@property
def rgb(self):
if self.__rgb is not None: return [*self.__rgb]
else: raise self.NotSet
@property
def cmyk(self):
if self.__cmyk is not None: return [*self.__cmyk]
else: raise self.NotSet
@property
def gray(self):
if self.__gray is not None: return [float(self.__gray)]
else: raise self.NotSet
@property
def custom(self):
if self.__custom is not None: return dict(self.__custom)
else: raise self.NotSet
@rgb.setter
def rgb(self, rgb):
self.__rgb = rgb
@cmyk.setter
def cmyk(self, cmyk):
self.__cmyk = cmyk
@gray.setter
def gray(self, color):
self.__gray = color
@custom.setter
def custom(self, color):
self.__custom = color
def copy(self):
cp = self.__class__()
cp.gray = self.__gray
cp.rgb = self.__rgb
cp.cmyk = self.__cmyk
cp.custom = self.__custom
return cp
def __repr__(self):
rep = "<PDFGraphicStateColor"
if self.__gray is not None: rep += f' gray={self.__gray}'
if self.__rgb is not None: rep += f' rgb=[{",".join(map(str, self.__rgb))}]'
if self.__cmyk is not None: rep += f' cmyk=[{",".join(map(str, self.__cmyk))}]'
if self.__custom is not None: rep += f' custom={self.__custom}'
return rep + '/>'
@@ -1,45 +0,0 @@
from .PDFGraphicStateColor import PDFGraphicStateColor
class PDFGraphicState(object):
def __init__(self):
self.linewidth = 0
self.linecap = None
self.linejoin = None
self.miterlimit = None
self.dash = None
self.intent = None
self.flatness = None
# stroking color
self.scolor = PDFGraphicStateColor()
# non stroking color
self.ncolor = PDFGraphicStateColor()
return
def copy(self):
obj = PDFGraphicState()
obj.linewidth = self.linewidth
obj.linecap = self.linecap
obj.linejoin = self.linejoin
obj.miterlimit = self.miterlimit
obj.dash = self.dash
obj.intent = self.intent
obj.flatness = self.flatness
obj.scolor = self.scolor.copy()
obj.ncolor = self.ncolor.copy()
return obj
def __repr__(self):
return ('<PDFGraphicState: linewidth=%r, linecap=%r, linejoin=%r, '
' miterlimit=%r, dash=%r, intent=%r, flatness=%r, '
' stroking color=%r, non stroking color=%r'
'>' %
(
self.linewidth, self.linecap, self.linejoin,
self.miterlimit, self.dash, self.intent, self.flatness,
str(self.scolor), str(self.ncolor)
)
)
-668
View File
@@ -1,668 +0,0 @@
import logging
import operator
from collections import OrderedDict
from typing import List
from .PDFColorSpace import PDFColorSpace
from .PDFColorSpace import PREDEFINED_COLORSPACE
from .cmapdb import CMapDB
from .cmapdb import CMap
from .PDFStream import PDFException
from .PDFStream.PDFObjRef import PDFObjRef
from .PDFStream import resolve1, list_value, dict_value
from .PDFStream import PDFStream
from .PSStackParser import PSEOF
from .PSStackParser import PSKeyword
from .PSStackParser import literal_name
from .PSStackParser import keyword_name
from .PSStackParser import PSStackParser
from .PSStackParser import LIT
from .PDFGraphicState import PDFGraphicState
from .PDFTextState import PDFTextState
from .PDFContentParser import PDFContentParser
from ..utils import mult_matrix, MATRIX_IDENTITY
from .constants import LITERAL_FORM, LITERAL_IMAGE
from .types import CurvePoint, CurvePath
log = logging.getLogger(__name__)
## Exceptions
##
class PDFInterpreterError(PDFException):
pass
class PDFPageInterpreter(object):
def __init__(self, rsrcmgr, device):
self.rsrcmgr = rsrcmgr
self.device = device
self.curpath: List[CurvePath] = []
return
def dup(self):
return self.__class__(self.rsrcmgr, self.device)
# init_resources(resources):
# Prepare the fonts and XObjects listed in the Resource attribute.
def init_resources(self, resources):
self.resources = resources
self.fontmap = {}
self.xobjmap = {}
self.csmap:OrderedDict = PREDEFINED_COLORSPACE.copy()
if not resources:
return
def get_colorspace(spec):
if isinstance(spec, list):
name = literal_name(spec[0])
else:
name = literal_name(spec)
if name == 'ICCBased' and isinstance(spec, list) and 2 <= len(spec):
return PDFColorSpace(name, PDFStream.validated_stream(spec[1])['N'])
elif name == 'DeviceN' and isinstance(spec, list) and 2 <= len(spec):
return PDFColorSpace(name, len(list_value(spec[1])))
else:
return PREDEFINED_COLORSPACE.get(name)
for (k, v) in iter(dict_value(resources).items()):
log.debug('Resource: %r: %r', k, v)
if k == 'Font':
for (fontid, spec) in iter(dict_value(v).items()):
objid = None
if isinstance(spec, PDFObjRef):
objid = spec.objid
spec = dict_value(spec)
self.fontmap[fontid] = self.rsrcmgr.get_font(objid, spec)
elif k == 'ColorSpace':
for (csid, spec) in iter(dict_value(v).items()):
self.csmap[csid] = get_colorspace(resolve1(spec))
elif k == 'ProcSet':
self.rsrcmgr.get_procset(list_value(v))
elif k == 'XObject':
for (xobjid, xobjstrm) in iter(dict_value(v).items()):
self.xobjmap[xobjid] = xobjstrm
return
# init_state(ctm)
# Initialize the text and graphic states for rendering a page.
def init_state(self, ctm):
# gstack: stack for graphical states.
self.gstack = []
self.ctm = ctm
self.device.set_ctm(self.ctm)
self.textstate = PDFTextState()
self.graphicstate = PDFGraphicState()
self.curpath = []
# argstack: stack for command arguments.
self.argstack = []
# set some global states.
self.scs = self.ncs = None
if self.csmap:
self.scs = self.ncs = next(iter(self.csmap.values()))
return
def push(self, obj):
self.argstack.append(obj)
return
def pop(self, n):
if n == 0:
return []
x = self.argstack[-n:]
self.argstack = self.argstack[:-n]
return x
def get_current_state(self):
return (self.ctm, self.textstate.copy(), self.graphicstate.copy())
def set_current_state(self, state):
(self.ctm, self.textstate, self.graphicstate) = state
self.device.set_ctm(self.ctm)
return
# gsave
def do_q(self):
self.gstack.append(self.get_current_state())
return
# grestore
def do_Q(self):
if self.gstack:
self.set_current_state(self.gstack.pop())
return
# concat-matrix
def do_cm(self, a1, b1, c1, d1, e1, f1):
self.ctm = mult_matrix((a1, b1, c1, d1, e1, f1), self.ctm)
self.device.set_ctm(self.ctm)
return
# setlinewidth
def do_w(self, linewidth):
self.graphicstate.linewidth = linewidth
return
# setlinecap
def do_J(self, linecap):
self.graphicstate.linecap = linecap
return
# setlinejoin
def do_j(self, linejoin):
self.graphicstate.linejoin = linejoin
return
# setmiterlimit
def do_M(self, miterlimit):
self.graphicstate.miterlimit = miterlimit
return
# setdash
def do_d(self, dash, phase):
self.graphicstate.dash = (dash, phase)
return
# setintent
def do_ri(self, intent):
self.graphicstate.intent = intent
return
# setflatness
def do_i(self, flatness):
self.graphicstate.flatness = flatness
return
# load-gstate
def do_gs(self, name):
#XXX
return
# moveto
def do_m(self, x: float, y: float):
self.curpath.append(CurvePath(CurvePath.METHOD.MOVE_TO, CurvePoint(x, y)))
return
# lineto
def do_l(self, x: float, y: float):
self.curpath.append(CurvePath(CurvePath.METHOD.LINE_TO, CurvePoint(x, y)))
return
# curveto
def do_c(self, x1: float, y1: float, x2: float, y2: float, x3: float, y3: float):
self.curpath.append(CurvePath(
CurvePath.METHOD.CURVE_BOTH_TO,
CurvePoint(x1, y1),
CurvePoint(x2, y2),
CurvePoint(x3, y3),
))
return
# urveto
def do_v(self, x2: float, y2: float, x3: float, y3: float):
self.curpath.append(CurvePath(
CurvePath.METHOD.CURVE_NEXT_TO,
CurvePoint(x2, y2),
CurvePoint(x3, y3),
))
return
# rveto
def do_y(self, x1: float, y1: float, x3: float, y3: float):
self.curpath.append(CurvePath(
CurvePath.METHOD.CURVE_FIRST_TO,
CurvePoint(x1, y1),
CurvePoint(x3, y3)
))
return
# closepath
def do_h(self):
self.curpath.append(CurvePath(CurvePath.METHOD.CLOSE_PATH))
return
# rectangle
def do_re(self, x, y, w, h):
self.curpath.append(CurvePath(CurvePath.METHOD.MOVE_TO, CurvePoint(x, y) ))
self.curpath.append(CurvePath(CurvePath.METHOD.LINE_TO, CurvePoint(x+w, y) ))
self.curpath.append(CurvePath(CurvePath.METHOD.LINE_TO, CurvePoint(x+w, y+h) ))
self.curpath.append(CurvePath(CurvePath.METHOD.LINE_TO, CurvePoint(x, y+h) ))
self.curpath.append(CurvePath(CurvePath.METHOD.CLOSE_PATH))
return
# stroke
def do_S(self):
self.device.paint_path(self.graphicstate.copy(), False, self.curpath)
self.curpath = []
return
# close-and-stroke
def do_s(self):
self.do_h()
self.do_S()
return
# fill
def do_f(self):
self.device.paint_path(self.graphicstate.copy(), False, self.curpath)
self.curpath = []
return
# fill (obsolete)
do_F = do_f
# fill-even-odd
def do_f_a(self):
self.device.paint_path(self.graphicstate.copy(), True, self.curpath)
self.curpath = []
return
# fill-and-stroke
def do_B(self):
self.device.paint_path(self.graphicstate.copy(), False, self.curpath)
self.curpath = []
return
# fill-and-stroke-even-odd
def do_B_a(self):
self.device.paint_path(self.graphicstate.copy(), True, self.curpath)
self.curpath = []
return
# close-fill-and-stroke
def do_b(self):
self.do_h()
self.do_B()
return
# close-fill-and-stroke-even-odd
def do_b_a(self):
self.do_h()
self.do_B_a()
return
# close-only
def do_n(self):
self.curpath = []
return
# clip
def do_W(self):
return
# clip-even-odd
def do_W_a(self):
return
# setcolorspace-stroking
def do_CS(self, name):
try:
self.scs = self.csmap[literal_name(name)]
except KeyError:
raise PDFInterpreterError('Undefined ColorSpace: %r' % name)
return
# setcolorspace-non-strokine
def do_cs(self, name):
try:
self.ncs = self.csmap[literal_name(name)]
except KeyError:
raise PDFInterpreterError('Undefined ColorSpace: %r' % name)
return
# setgray-stroking
def do_G(self, gray):
self.graphicstate.scolor.gray = gray
#self.do_CS(LITERAL_DEVICE_GRAY)
return
# setgray-non-stroking
def do_g(self, gray):
self.graphicstate.ncolor.gray = gray
#self.do_cs(LITERAL_DEVICE_GRAY)
return
# setrgb-stroking
def do_RG(self, r, g, b):
self.graphicstate.scolor.rgb = (r*255, g*255, b*255)
#self.do_CS(LITERAL_DEVICE_RGB)
return
# setrgb-non-stroking
def do_rg(self, r, g, b):
self.graphicstate.ncolor.rgb = (r*255, g*255, b*255)
#self.do_cs(LITERAL_DEVICE_RGB)
return
# setcmyk-stroking
def do_K(self, c, m, y, k):
self.graphicstate.scolor.cmyk = (c, m, y, k)
#self.do_CS(LITERAL_DEVICE_CMYK)
return
# setcmyk-non-stroking
def do_k(self, c, m, y, k):
self.graphicstate.ncolor.cmyk = (c, m, y, k)
#self.do_cs(LITERAL_DEVICE_CMYK)
return
def __process_ICCBased(self, n: int, components: list):
if n == 1:
return {
"type": "gray",
"definition": components
}
elif n == 3:
return {
"type": "rgb",
"definition": components
}
elif n == 4:
return {
"type": "cmyk",
"definition": components
}
raise PDFInterpreterError("Unsupported ICCBased color")
# setcolor
def __process_scn(self):
if self.ncs:
n = self.ncs.ncomponents
else:
raise PDFInterpreterError('No colorspace specified!')
components = self.pop(n)
obj_id = components[0]
if self.ncs.name == "ICCBased":
return self.__process_ICCBased(n, components)
if self.ncs.name not in self.resources.keys():
log.warning("Color space not in resources")
return {
"type": self.ncs.name,
"defintion": f"{n}|({','.join(map(str, components))})"
}
if obj_id.name not in self.resources[self.ncs.name].keys():
raise PDFInterpreterError("Invalid Color reference")
color_data = self.resources[self.ncs.name][obj_id.name].resolve()
if isinstance(color_data, PDFStream):
color_data = color_data.attrs
return {
"type": self.ncs.name,
"defintion": color_data
}
def do_SCN(self):
processed_col = self.__process_scn()
if processed_col['type'] == 'rgb':
self.graphicstate.scolor.rgb = processed_col['definition']
elif processed_col['type'] == 'gray':
self.graphicstate.scolor.gray = processed_col['definition'][0]
elif processed_col['type'] == 'cmyk':
self.graphicstate.scolor.cmyk = processed_col['definition']
else:
self.graphicstate.scolor.custom = processed_col
return
def do_scn(self):
processed_col = self.__process_scn()
if processed_col['type'] == 'rgb':
self.graphicstate.ncolor.rgb = processed_col['definition']
elif processed_col['type'] == 'gray':
self.graphicstate.ncolor.gray = processed_col['definition'][0]
elif processed_col['type'] == 'cmyk':
self.graphicstate.ncolor.cmyk = processed_col['definition']
else:
self.graphicstate.ncolor.custom = processed_col
return
def do_SC(self):
self.do_SCN()
return
def do_sc(self):
self.do_scn()
return
# sharing-name
def do_sh(self, name):
return
# begin-text
def do_BT(self):
self.textstate.reset()
return
# end-text
def do_ET(self):
return
# begin-compat
def do_BX(self):
return
# end-compat
def do_EX(self):
return
# marked content operators
def do_MP(self, tag):
self.device.do_tag(tag)
return
def do_DP(self, tag, props):
self.device.do_tag(tag, props)
return
def do_BMC(self, tag):
self.device.begin_tag(tag)
return
def do_BDC(self, tag, props):
self.device.begin_tag(tag, props)
return
def do_EMC(self):
self.device.end_tag()
return
# setcharspace
def do_Tc(self, space):
self.textstate.charspace = space
return
# setwordspace
def do_Tw(self, space):
self.textstate.wordspace = space
return
# def do_Th(self, )
# textscale
def do_Tz(self, scale):
self.textstate.scaling = scale
return
# setleading
def do_TL(self, leading):
self.textstate.leading = -leading
return
# selectfont
def do_Tf(self, fontid, fontsize):
try:
self.textstate.font = self.fontmap[literal_name(fontid)]
except KeyError:
raise PDFInterpreterError('Undefined Font id: %r' % fontid)
# self.textstate.font = self.rsrcmgr.get_font(None, {})
self.textstate.fontsize = fontsize
return
# setrendering
def do_Tr(self, render):
self.textstate.render = render
return
# settextrise
def do_Ts(self, rise):
self.textstate.rise = rise
return
# text-move
def do_Td(self, tx, ty):
(a, b, c, d, e, f) = self.textstate.matrix
self.textstate.matrix = (a, b, c, d, tx*a+ty*c+e, tx*b+ty*d+f)
self.textstate.linematrix = [0, 0]
return
# text-move
def do_TD(self, tx, ty):
(a, b, c, d, e, f) = self.textstate.matrix
self.textstate.matrix = (a, b, c, d, tx*a+ty*c+e, tx*b+ty*d+f)
self.textstate.leading = ty
self.textstate.linematrix = [0, 0]
return
# textmatrix
def do_Tm(self, a, b, c, d, e, f):
self.textstate.matrix = (a, b, c, d, e, f)
self.textstate.linematrix = [0, 0]
return
# nextline
def do_T_a(self):
(a, b, c, d, e, f) = self.textstate.matrix
self.textstate.matrix = (a, b, c, d, self.textstate.leading*c+e, self.textstate.leading*d+f)
self.textstate.linematrix = [0, 0]
return
# show-pos
def do_TJ(self, seq: bytearray):
if self.textstate.font is None:
raise PDFInterpreterError('No font specified!')
self.device.render_string(self.textstate.copy(), seq, self.ncs, self.graphicstate.copy())
return
# show
def do_Tj(self, s):
self.do_TJ([s])
return
# quote
def do__q(self, s):
self.do_T_a()
self.do_TJ([s])
return
# doublequote
def do__w(self, aw, ac, s):
self.do_Tw(aw)
self.do_Tc(ac)
self.do_TJ([s])
return
# inline image
def do_BI(self): # never called
return
def do_ID(self): # never called
return
def do_EI(self, obj: PDFStream):
if 'W' in obj and 'H' in obj:
iobjid = str(id(obj))
self.device.begin_figure(iobjid, (0, 0, 1, 1), MATRIX_IDENTITY)
self.device.render_image(iobjid, obj)
self.device.end_figure(iobjid)
return
# invoke an XObject
def do_Do(self, xobjid):
xobjid = literal_name(xobjid)
try:
xobj = PDFStream.validated_stream(self.xobjmap[xobjid])
except KeyError:
raise PDFInterpreterError('Undefined xobject id: %r' % xobjid)
log.info('Processing xobj: %r', xobj)
subtype = xobj.get('Subtype')
if subtype is LITERAL_FORM and 'BBox' in xobj:
interpreter = self.dup()
bbox = list_value(xobj['BBox'])
matrix = list_value(xobj.get('Matrix', MATRIX_IDENTITY))
# According to PDF reference 1.7 section 4.9.1, XObjects in
# earlier PDFs (prior to v1.2) use the page's Resources entry
# instead of having their own Resources entry.
xobjres = xobj.get('Resources')
resources = dict_value(xobjres) if xobjres else self.resources.copy()
self.device.begin_figure(xobjid, bbox, matrix)
interpreter.render_contents(resources, [xobj], ctm=mult_matrix(matrix, self.ctm))
self.device.end_figure(xobjid)
elif subtype is LITERAL_IMAGE and 'Width' in xobj and 'Height' in xobj:
self.device.begin_figure(xobjid, (0, 0, 1, 1), MATRIX_IDENTITY)
self.device.render_image(xobjid, xobj)
self.device.end_figure(xobjid)
else:
# unsupported xobject type.
pass
return
def process_page(self, page):
log.info('Processing page: %r', page)
(x0, y0, x1, y1) = page.mediabox
if page.rotate == 90:
ctm = (0, -1, 1, 0, -y0, x1)
elif page.rotate == 180:
ctm = (-1, 0, 0, -1, x1, y1)
elif page.rotate == 270:
ctm = (0, 1, -1, 0, y1, -x0)
else:
ctm = (1, 0, 0, 1, -x0, -y0)
self.device.begin_page(page, ctm)
self.render_contents(page.resources, page.contents, ctm=ctm)
return self.device.end_page(page)
# render_contents(resources, streams, ctm)
# Render the content streams.
# This method may be called recursively.
def render_contents(self, resources, streams, ctm=MATRIX_IDENTITY):
log.info('render_contents: resources=%r, streams=%r, ctm=%r',
resources, streams, ctm)
self.init_resources(resources)
self.init_state(ctm)
self.execute(list_value(streams))
return
def execute(self, streams):
try:
parser = PDFContentParser(streams)
except PSEOF:
# empty page
return
while True:
try:
(_, obj) = parser.nextobject()
except PSEOF:
break
if isinstance(obj, PSKeyword):
name = keyword_name(obj)
method = 'do_%s' % name.replace('*', '_a').replace('"', '_w').replace("'", '_q')
if hasattr(self, method):
func = getattr(self, method)
nargs = func.__code__.co_argcount-1
if nargs:
args = self.pop(nargs)
if len(args) == nargs:
func(*args)
else:
log.debug('exec: %s', name)
func()
else:
raise PDFInterpreterError('Unknown operator: %r' % name)
else:
self.push(obj)
return
-103
View File
@@ -1,103 +0,0 @@
import logging
from .cmapdb import CMapDB, CMap
from .PDFFont import PDFFontError
from .PDFFont import PDFType1Font
from .PDFFont import PDFTrueTypeFont
from .PDFFont import PDFType3Font
from .PDFFont import PDFCIDFont
from .PDFStream import PDFException
from .PDFStream import resolve1, dict_value, list_value
from .constants import LITERAL_PDF, LITERAL_TEXT, LITERAL_FONT
from .PSStackParser import literal_name, settings
log = logging.getLogger(__name__)
## Exceptions
##
class PDFResourceError(PDFException):
pass
class PDFInterpreterError(PDFException):
pass
## Resource Manager
##
class PDFResourceManager(object):
"""Repository of shared resources.
ResourceManager facilitates reuse of shared resources
such as fonts and images so that large objects are not
allocated multiple times.
"""
def __init__(self, caching=True):
self.caching = caching
self._cached_fonts = {}
return
def get_procset(self, procs):
for proc in procs:
if proc is LITERAL_PDF:
pass
elif proc is LITERAL_TEXT:
pass
else:
#raise PDFResourceError('ProcSet %r is not supported.' % proc)
pass
return
def get_cmap(self, cmapname, strict=False):
try:
return CMapDB.get_cmap(cmapname)
except CMapDB.CMapNotFound:
if strict:
raise
return CMap()
def get_font(self, objid: int, spec: dict):
if objid and objid in self._cached_fonts:
font = self._cached_fonts[objid]
else:
log.info('get_font: create: objid=%r, spec=%r', objid, spec)
if settings.STRICT:
if spec['Type'] is not LITERAL_FONT:
raise PDFFontError('Type is not /Font')
# Create a Font object.
if 'Subtype' in spec:
subtype = literal_name(spec['Subtype'])
else:
if settings.STRICT:
raise PDFFontError('Font Subtype is not specified.')
subtype = 'Type1'
if subtype in ('Type1', 'MMType1'):
# Type1 Font
font = PDFType1Font(self, spec)
elif subtype == 'TrueType':
# TrueType Font
font = PDFTrueTypeFont(self, spec)
elif subtype == 'Type3':
# Type3 Font
font = PDFType3Font(self, spec)
elif subtype in ('CIDFontType0', 'CIDFontType2'):
# CID Font
font = PDFCIDFont(self, spec)
elif subtype == 'Type0':
# Type0 Font
dfonts = list_value(spec['DescendantFonts'])
assert dfonts
subspec = dict_value(dfonts[0]).copy()
for k in ('Encoding', 'ToUnicode'):
if k in spec:
subspec[k] = resolve1(spec[k])
font = self.get_font(None, subspec)
else:
if settings.STRICT:
raise PDFFontError('Invalid Font spec: %r' % spec)
font = PDFType1Font(self, spec) # this is so wrong!
if objid and self.caching:
self._cached_fonts[objid] = font
return font
+82
View File
@@ -0,0 +1,82 @@
from pdfmajor.parser.PDFPage import PDFPage
from pdfmajor.parser.PDFStream import list_value
from pdfmajor.parser.PDFContentParser import PDFContentParser
from pdfmajor.parser.PSStackParser import PSKeyword
from pdfmajor.parser.PSStackParser import keyword_name
from .commands import PDFStateStack, PDFCommands
from .commands import LTXObject
from .utils import init_resources
from .XObjectInterpreter import XObjectInterpreter
class PageInterpreter:
class PDFInterpreterError(Exception): pass
def __init__(self, page: PDFPage, page_num: int, font_cache: dict = {}):
(x0, y0, x1, y1) = page.mediabox
if page.rotate == 90:
ctm = [0, -1, 1, 0, -y0, x1]
elif page.rotate == 180:
ctm = [-1, 0, 0, -1, x1, y1]
elif page.rotate == 270:
ctm = [0, 1, -1, 0, y1, -x0]
else:
ctm = [1, 0, 0, 1, -x0, -y0]
# Locals
self.font_cache = font_cache
self.page = page
self.page_num = page_num
self.height = y1-y0
self.width = x1-x0
# Init State
self.state: PDFStateStack = PDFStateStack()
self.state.t_matrix = ctm
self.state.resources = page.resources
# set some global states.
self.state.graphics.ncolspace = self.state.graphics.scolspace = None
if self.state.colorspace_map:
col_space = next(iter(self.state.colorspace_map.values()))
self.state.graphics.ncolspace = self.state.graphics.scolspace = col_space
# Init Resources
init_resources(self.state, self.font_cache)
def __iter__(self):
parser = PDFContentParser(list_value(self.page.contents))
history = []
for obj in parser:
history.append(obj)
if isinstance(obj, PSKeyword):
name = keyword_name(obj)
method = name.replace('*', '_a').replace('"', '_w').replace("'", '_q')
if method in PDFCommands.commands.keys():
func = PDFCommands.commands.get(method)
nargs = func.__code__.co_argcount-1
if nargs:
args = self.state.pop(nargs)
if len(args) == nargs:
func(self.state, *args)
else:
raise self.PDFInterpreterError(f"Invalid Number of Args provided {method}")
else:
# log.debug('exec: %s', name)
func(self.state)
else:
raise self.PDFInterpreterError('Unknown operator: %r' % name)
else:
self.state.argstack.append(obj)
for complete_item in self.state.complete_layout_items:
if isinstance(complete_item, LTXObject):
interpeter = XObjectInterpreter(
streams=[complete_item.stream],
resources=complete_item.resources,
ctm=complete_item.t_matrix
)
for item in interpeter:
complete_item.add(item)
yield complete_item
self.state.complete_layout_items = []
def __repr__(self) -> str:
return f"<Page:{self.page_num}/>"
@@ -0,0 +1,70 @@
from pdfmajor.utils import MATRIX_IDENTITY
from pdfmajor.parser.PSStackParser import literal_name
from pdfmajor.parser.PDFParser import PDFStream
from pdfmajor.parser.PSStackParser import PSEOF
from pdfmajor.parser.PSStackParser import PSKeyword
from pdfmajor.parser.PSStackParser import keyword_name
from pdfmajor.parser.PSStackParser import PSStackParser
from pdfmajor.parser.constants import LITERAL_PDF, LITERAL_TEXT, LITERAL_FONT
from pdfmajor.parser.PDFStream import resolve1, list_value, dict_value
from pdfmajor.parser.PDFStream.PDFObjRef import PDFObjRef
from pdfmajor.parser.PDFContentParser import PDFContentParser
from .commands.state import PDFStateStack, PDFColorSpace, PREDEFINED_COLORSPACE
from .commands import PDFCommands, LTXObject
from .utils import init_resources
class XObjectInterpreter:
class PDFInterpreterError(Exception): pass
def __init__(self, streams, resources, ctm):
self.streams = streams
self.font_cache = {}
self.state = PDFStateStack()
self.state.t_matrix = ctm
self.state.resources = resources
# set some global states.
self.state.graphics.ncolspace = self.state.graphics.scolspace = None
if self.state.colorspace_map:
col_space = next(iter(self.state.colorspace_map.values()))
self.state.graphics.ncolspace = self.state.graphics.scolspace = col_space
init_resources(self.state, self.font_cache)
def __iter__(self):
parser = PDFContentParser(self.streams)
for obj in parser:
if isinstance(obj, PSKeyword):
name = keyword_name(obj)
method = name.replace('*', '_a').replace('"', '_w').replace("'", '_q')
if method in PDFCommands.commands.keys():
func = PDFCommands.commands.get(method)
nargs = func.__code__.co_argcount-1
if nargs:
args = self.state.pop(nargs)
if len(args) == nargs:
func(self.state, *args)
else:
raise self.PDFInterpreterError(f"Invalid Number of Args provided {method}")
else:
# log.debug('exec: %s', name)
func(self.state)
else:
raise self.PDFInterpreterError('Unknown operator: %r' % name)
else:
self.state.argstack.append(obj)
for complete_item in self.state.complete_layout_items:
if isinstance(complete_item, LTXObject):
interpeter = self.__class__(
streams=[complete_item.stream],
resources=complete_item.resources,
ctm=complete_item.t_matrix
)
for item in interpeter:
complete_item.add(item)
yield complete_item
self.state.complete_layout_items = []
+72
View File
@@ -0,0 +1,72 @@
from typing import List
from tqdm import tqdm
from ..parser.PDFPage import PDFPage
from ..utils import set_log_level, get_logger, logging
from .commands.state import PDFStateStack, PDFColorSpace, PREDEFINED_COLORSPACE
from .commands import PDFCommands
from .commands import LTCharBlock, LTChar
from .commands import LTCurve, LTLine, LTHorizontalLine, LTVerticalLine, LTRect
from .commands import LTImage
from .commands import LTXObject
from .PageInterpreter import PageInterpreter
log = get_logger(__name__)
class PDFInterpreter:
class AccessError(Exception): pass
def __init__(self,
input_file_path: str,
maxpages: int = 0,
caching: bool = True,
check_extractable: bool = True,
password: str = None,
pagenos: List[int] = None,
preload: bool = False,
debug_level: int = logging.WARNING,
):
self.input_file_path = input_file_path
self.maxpages = maxpages
self.caching = caching
self.check_extractable = check_extractable
self.password = password
self.pagenos = pagenos
self.debug_level = debug_level
self.__pages = []
if preload:
for _ in self.__load_pages():
pass
def __load_pages(self):
set_log_level(self.debug_level)
log.info("Opening file...")
with open(self.input_file_path, 'rb') as input_file:
font_cache = {}
log.info("Parsing file...")
pages = PDFPage.get_pages(
input_file,
pagenos=self.pagenos,
maxpages=self.maxpages,
password=self.password,
caching=self.caching,
check_extractable=self.check_extractable
)
for page_num, page in tqdm(enumerate(pages), desc="Reading pages...", disable=self.debug_level > logging.INFO):
self.__pages.append(PageInterpreter(page, page_num, font_cache))
yield self.__pages[-1]
log.info(f"Done Reading {len(self.__pages)} pages.")
set_log_level(logging.WARNING)
if len(self.__pages) == 0:
raise self.AccessError("No pages found in pdf-file")
def __iter__(self):
if len(self.__pages) > 0:
set_log_level(self.debug_level)
for page in tqdm(self.__pages, desc="Reading pages from store...", disable=self.debug_level > logging.INFO):
yield page
set_log_level(logging.WARNING)
else:
for page in self.__load_pages():
yield page
@@ -0,0 +1,30 @@
from typing import List
from pdfmajor.utils import MATRIX_IDENTITY, mult_matrix
from pdfmajor.parser.PSStackParser import literal_name
from pdfmajor.parser.PDFStream import PDFStream, list_value, dict_value
from pdfmajor.parser.constants import LITERAL_FORM, LITERAL_IMAGE
from .state import PDFStateStack, PDFColorSpace
from .state.Curves import CurveMethod, CurvePath, CurvePoint
class PDFCommands:
commands = {}
class InvalidOperation(Exception): pass
class RepeatedCommand(Exception): pass
@classmethod
def __add(cls, cmd_name: str, cmd: callable):
if not (cmd_name in cls.commands.keys()):
cls.commands[cmd_name] = cmd
return cmd
raise cls.RepeatedCommand(cmd_name)
@classmethod
def add(cls, *cmd_names: List[str]):
def wrap(cmd: callable):
for cmd_name in cmd_names:
cls.__add(cmd_name, cmd)
return cmd
return wrap
@@ -0,0 +1,9 @@
from .commands import PDFCommands
from .state import PDFStateStack, PDFGraphicState, PDFTextState
from .state.PDFGraphicState import PDFColor, PDFColorSpace, PREDEFINED_COLORSPACE
from .state import PDFFont, get_font
from .state import LTCharBlock, LTChar
from .state import LTCurve, LTLine, LTHorizontalLine, LTVerticalLine, LTRect
from .state import LTImage
from .state import LTXObject
from .state import LTItem, LTComponent, LTContainer
+486
View File
@@ -0,0 +1,486 @@
from pdfmajor.utils import mult_matrix, MATRIX_IDENTITY, apply_matrix_pt
from pdfmajor.parser.PSStackParser import literal_name
from pdfmajor.parser.PDFStream import PDFStream, list_value, dict_value
from pdfmajor.parser.constants import LITERAL_FORM, LITERAL_IMAGE
from .PDFCommands import PDFCommands
from .state import PDFStateStack, PDFGraphicState
from .state.Curves import CurveMethod, CurvePath, CurvePoint
# from .state.PDFItem import PDFImage, PDFShape, PDFText, PDFXObject
from .state import make_char_block, make_curve, make_image, make_xobject
# No Support
@PDFCommands.add('W','W_a', 'sh', 'ET', 'BX', 'EX' ,'BI', 'ID', 'gs')
def do_no_support(stack: PDFStateStack) -> PDFStateStack:
return stack
# marked content operators
@PDFCommands.add('MP', 'DP')
def do_MP(stack: PDFStateStack) -> PDFStateStack:
# self.device.do_tag(tag)
return stack
@PDFCommands.add('BMC', 'BDC')
def do_BMC(stack: PDFStateStack) -> PDFStateStack:
# self.device.begin_tag(tag)
return stack
@PDFCommands.add('EMC')
def do_EMC(stack: PDFStateStack) -> PDFStateStack:
# self.device.end_tag(tag)
return stack
@PDFCommands.add('q')
def do_q(stack: PDFStateStack) -> PDFStateStack:
stack.gstack.append([
stack.t_matrix,
stack.text.copy(),
stack.graphics.copy()
])
return stack
@PDFCommands.add('Q')
def do_Q(stack: PDFStateStack) -> PDFStateStack:
if len(stack.gstack) > 0:
stack.t_matrix, stack.text, stack.graphicstate = stack.gstack.pop()
return stack
# concat-matrix
@PDFCommands.add('cm')
def do_cm(stack: PDFStateStack, a1, b1, c1, d1, e1, f1) -> PDFStateStack:
stack.t_matrix = mult_matrix((a1, b1, c1, d1, e1, f1), stack.t_matrix)
return stack
# setlinewidth
@PDFCommands.add('w')
def do_w(stack: PDFStateStack, linewidth) -> PDFStateStack:
stack.graphics.linewidth = linewidth
return stack
# setlinecap
@PDFCommands.add('J')
def do_J(stack: PDFStateStack, linecap) -> PDFStateStack:
stack.graphics.linecap = linecap
return stack
# setlinejoin
@PDFCommands.add('j')
def do_j(stack: PDFStateStack, linejoin) -> PDFStateStack:
stack.graphics.linejoin = linejoin
return stack
# setmiterlimit
@PDFCommands.add('M')
def do_M(stack: PDFStateStack, miterlimit) -> PDFStateStack:
stack.graphics.miterlimit = miterlimit
return stack
# setdash
@PDFCommands.add('d')
def do_d(stack: PDFStateStack, dash, phase) -> PDFStateStack:
stack.graphics.dash = (dash, phase)
return stack
# setintent
@PDFCommands.add('ri')
def do_ri(stack: PDFStateStack, intent) -> PDFStateStack:
stack.graphics.intent = intent
return stack
# setflatness
@PDFCommands.add('i')
def do_i(stack: PDFStateStack, flatness) -> PDFStateStack:
stack.graphics.flatness = flatness
return stack
# moveto
@PDFCommands.add('m')
def do_m(stack: PDFStateStack, x: float, y: float) -> PDFStateStack:
stack.curvestacks.append(CurvePath(
CurvePath.METHOD.MOVE_TO,
CurvePoint(x, y)
))
return stack
# lineto
@PDFCommands.add('l')
def do_l(stack: PDFStateStack, x: float, y: float) -> PDFStateStack:
stack.curvestacks.append(CurvePath(
CurvePath.METHOD.LINE_TO,
CurvePoint(x, y)
))
return stack
# curveto
@PDFCommands.add('c')
def do_c(stack: PDFStateStack,
x1: float, y1: float,
x2: float, y2: float,
x3: float, y3: float ) -> PDFStateStack:
stack.curvestacks.append(CurvePath(
CurvePath.METHOD.CURVE_BOTH_TO,
CurvePoint(x1, y1),
CurvePoint(x2, y2),
CurvePoint(x3, y3),
))
return stack
# urveto
@PDFCommands.add('v')
def do_v(stack: PDFStateStack,
x2: float, y2: float,
x3: float, y3: float) -> PDFStateStack:
stack.curvestacks.append(CurvePath(
CurvePath.METHOD.CURVE_NEXT_TO,
CurvePoint(x2, y2),
CurvePoint(x3, y3),
))
return stack
# rveto
@PDFCommands.add('y')
def do_y(stack: PDFStateStack,
x1: float, y1: float,
x3: float, y3: float) -> PDFStateStack:
stack.curvestacks.append(CurvePath(
CurvePath.METHOD.CURVE_FIRST_TO,
CurvePoint(x1, y1),
CurvePoint(x3, y3)
))
return stack
# closepath
@PDFCommands.add('h')
def do_h(stack: PDFStateStack) -> PDFStateStack:
stack.curvestacks.append(CurvePath(CurvePath.METHOD.CLOSE_PATH))
return stack
# rectangle
@PDFCommands.add('re')
def do_re(stack: PDFStateStack, x, y, w, h) -> PDFStateStack:
stack.curvestacks.extend([
CurvePath(CurvePath.METHOD.MOVE_TO, CurvePoint(x, y) ),
CurvePath(CurvePath.METHOD.LINE_TO, CurvePoint(x+w, y) ),
CurvePath(CurvePath.METHOD.LINE_TO, CurvePoint(x+w, y+h) ),
CurvePath(CurvePath.METHOD.LINE_TO, CurvePoint(x, y+h) ),
CurvePath(CurvePath.METHOD.CLOSE_PATH)
])
return stack
# stroke, fill
@PDFCommands.add('S', 'f', 'F', 'B')
def do_complete_path(stack: PDFStateStack) -> PDFStateStack:
stack.complete_layout_items.append(make_curve(
stack.t_matrix,
stack.graphics.copy(),
False,
stack.curvestacks
))
stack.curvestacks = []
return stack
# sroke, fill-even-odd
@PDFCommands.add('f_a', 'B_a')
def do_complete_path_evenodd(stack: PDFStateStack) -> PDFStateStack:
stack.complete_layout_items.append(make_curve(
stack.t_matrix,
stack.graphics.copy(),
True,
stack.curvestacks
))
stack.curvestacks = []
return stack
# close-and-stroke (do_h, do_S)
@PDFCommands.add('s', 'b')
def do_close_complete(stack: PDFStateStack) -> PDFStateStack:
stack.curvestacks.append(CurvePath(
CurvePath.METHOD.CLOSE_PATH
))
stack.complete_layout_items.append(make_curve(
stack.t_matrix,
stack.graphics.copy(),
False,
stack.curvestacks
))
stack.curvestacks = []
return stack
# close-fill-and-stroke-even-odd
@PDFCommands.add('b_a')
def do_close_complete_evenodd(stack: PDFStateStack) -> PDFStateStack:
stack.curvestacks.append(CurvePath(
CurvePath.METHOD.CLOSE_PATH
))
stack.complete_layout_items.append(make_curve(
stack.t_matrix,
stack.graphics.copy(),
True,
stack.curvestacks
))
stack.curvestacks = []
return stack
# close-only (empty path)
@PDFCommands.add('n')
def do_n(stack: PDFStateStack) -> PDFStateStack:
stack.curvestacks = []
return stack
# setcolorspace-stroking
@PDFCommands.add('CS')
def do_CS(stack: PDFStateStack, name: str) -> PDFStateStack:
try:
stack.graphics.scolspace = stack.colorspace_map[literal_name(name)]
return stack
except KeyError:
raise PDFCommands.InvalidOperation('Undefined ColorSpace: %r' % name)
# setcolorspace-non-strokine
@PDFCommands.add('cs')
def do_cs(stack: PDFStateStack, name: str) -> PDFStateStack:
try:
stack.graphics.ncolspace = stack.colorspace_map[literal_name(name)]
return stack
except KeyError:
raise PDFCommands.InvalidOperation('Undefined ColorSpace: %r' % name)
# setgray-stroking
@PDFCommands.add('G')
def do_G(stack: PDFStateStack, gray: float) -> PDFStateStack:
stack.graphics.set_stroke_color(
stack.colorspace_map['DeviceGray'],
gray
)
return stack
# setgray-non-stroking
@PDFCommands.add('g')
def do_g(stack: PDFStateStack, gray: float) -> PDFStateStack:
stack.graphics.set_nostroke_color(
stack.colorspace_map['DeviceGray'],
gray
)
return stack
# setrgb-stroking
@PDFCommands.add('RG')
def do_RG(stack: PDFStateStack, r, g, b) -> PDFStateStack:
stack.graphics.set_stroke_color(
stack.colorspace_map['DeviceRGB'],
round(r*255),
round(g*255),
round(b*255)
)
return stack
# setrgb-non-stroking
@PDFCommands.add('rg')
def do_rg(stack: PDFStateStack, r, g, b) -> PDFStateStack:
stack.graphics.set_nostroke_color(
stack.colorspace_map['DeviceRGB'],
round(r*255),
round(g*255),
round(b*255)
)
return stack
# setcmyk-stroking
@PDFCommands.add('K')
def do_K(stack: PDFStateStack, c, m, y, k) -> PDFStateStack:
stack.graphics.set_stroke_color(
stack.colorspace_map['DeviceCMYK'],
c, m, y, k
)
return stack
# setcmyk-non-stroking
@PDFCommands.add('k')
def do_k(stack: PDFStateStack, c, m, y, k) -> PDFStateStack:
stack.graphics.set_stroke_color(
stack.colorspace_map['DeviceCMYK'],
c, m, y, k
)
return stack
@PDFCommands.add('SCN','SC')
def do_SCN(stack: PDFStateStack) -> PDFStateStack:
if stack.graphics.scolspace is None:
raise PDFCommands.InvalidOperation('No colorspace specified!')
components = stack.pop(stack.graphics.scolspace.ncomponents)
stack.graphics.set_stroke_color(
stack.graphics.scolspace,
*components
)
return stack
@PDFCommands.add('scn','sc')
def do_scn(stack: PDFStateStack) -> PDFStateStack:
if stack.graphics.ncolspace is None:
raise PDFCommands.InvalidOperation('No colorspace specified!')
components = stack.pop(stack.graphics.ncolspace.ncomponents)
stack.graphics.set_nostroke_color(
stack.graphics.ncolspace,
*components
)
return stack
# begin-text
@PDFCommands.add('BT')
def do_BT(stack: PDFStateStack) -> PDFStateStack:
stack.text.matrix = MATRIX_IDENTITY
stack.text.linematrix = [0, 0]
return stack
# setcharspace
@PDFCommands.add('Tc')
def do_Tc(stack: PDFStateStack, charspace) -> PDFStateStack:
stack.text.charspace = charspace
return stack
# setwordspace
@PDFCommands.add('Tw')
def do_Tw(stack: PDFStateStack, wordspace) -> PDFStateStack:
stack.text.wordspace = wordspace
return stack
# textscale
@PDFCommands.add('Tz')
def do_Tz(stack: PDFStateStack, scaling) -> PDFStateStack:
stack.text.scaling = scaling
return stack
# setleading
@PDFCommands.add('TL')
def do_TL(stack: PDFStateStack, leading) -> PDFStateStack:
stack.text.leading = -leading
return stack
# selectfont
@PDFCommands.add('Tf')
def do_Tf(stack: PDFStateStack, fontid, fontsize) -> PDFStateStack:
stack.text.fontsize = fontsize
try:
stack.text.font = stack.fontmap[literal_name(fontid)]
return stack
except KeyError:
raise PDFCommands.InvalidOperation('Undefined Font id: %r' % fontid)
# setrendering
@PDFCommands.add('Tr')
def do_Tr(stack: PDFStateStack, render) -> PDFStateStack:
stack.text.render = render
return stack
# settextrise
@PDFCommands.add('Ts')
def do_Ts(stack: PDFStateStack, rise) -> PDFStateStack:
stack.text.rise = rise
return stack
# text-move
@PDFCommands.add('Td')
def do_Td(stack: PDFStateStack, tx, ty) -> PDFStateStack:
(a, b, c, d, e, f) = stack.text.matrix
stack.text.matrix = (a, b, c, d, tx*a+ty*c+e, tx*b+ty*d+f)
stack.text.linematrix = [0, 0]
return stack
# text-move
@PDFCommands.add('TD')
def do_TD(stack: PDFStateStack, tx, ty) -> PDFStateStack:
(a, b, c, d, e, f) = stack.text.matrix
stack.text.matrix = (a, b, c, d, tx*a+ty*c+e, tx*b+ty*d+f)
stack.text.leading = ty
stack.text.linematrix = [0, 0]
return stack
# textmatrix
@PDFCommands.add('Tm')
def do_Tm(stack: PDFStateStack, a, b, c, d, e, f) -> PDFStateStack:
stack.text.matrix = (a, b, c, d, e, f)
stack.text.linematrix = [0, 0]
return stack
# nextline
@PDFCommands.add('T_a')
def do_T_a(stack: PDFStateStack) -> PDFStateStack:
(a, b, c, d, e, f) = stack.text.matrix
stack.text.matrix = (a, b, c, d, stack.text.leading*c+e, stack.text.leading*d+f)
stack.text.linematrix = [0, 0]
return stack
# show-pos
@PDFCommands.add('TJ')
def do_TJ(stack: PDFStateStack, seq: bytearray) -> PDFStateStack:
if stack.text.font is None:
raise PDFCommands.InvalidOperation("No Font Specified")
stack.complete_layout_items.append(make_char_block(
seq,
stack.t_matrix,
stack.text.copy(),
stack.graphics.ncolor.copy()
))
return stack
# show
@PDFCommands.add('Tj')
def do_Tj(stack: PDFStateStack, b: bytes) -> PDFStateStack:
return do_TJ(stack, [b])
# quote
@PDFCommands.add('_q')
def do__q(stack: PDFStateStack, b: bytes) -> PDFStateStack:
return do_TJ(do_T_a(stack), [b])
# doublequote
@PDFCommands.add('_w')
def do__w(stack: PDFStateStack, wordspace, charspace, b) -> PDFStateStack:
stack.text.wordspace = wordspace
stack.text.charspace = charspace
return do_TJ(stack, [b])
@PDFCommands.add('EI')
def do_EI(stack: PDFStateStack, obj) -> PDFStateStack:
if 'W' in obj and 'H' in obj:
stack.complete_layout_items.append(make_image(
obj, stack.ctm
))
return stack
# invoke an XObject
@PDFCommands.add('Do')
def do_Do(stack: PDFStateStack, xobjid) -> PDFStateStack:
xobjid = literal_name(xobjid)
try:
xobj = PDFStream.validated_stream(stack.xobjmap[xobjid])
except KeyError:
raise PDFCommands.InvalidOperation('Undefined xobject id: %r' % xobjid)
# log.info('Processing xobj: %r', xobj)
subtype = xobj.get('Subtype')
if subtype is LITERAL_FORM and 'BBox' in xobj:
bbox = list_value(xobj['BBox'])
matrix = list_value(xobj.get('Matrix', MATRIX_IDENTITY))
# According to PDF reference 1.7 section 4.9.1, XObjects in
# earlier PDFs (prior to v1.2) use the page's Resources entry
# instead of having their own Resources entry.
xobjres = xobj.get('Resources')
resources = dict_value(xobjres) if xobjres else stack.resources.copy()
stack.complete_layout_items.append(make_xobject(
obj=xobj,
bbox=bbox,
ctm=stack.t_matrix,
matrix=matrix,
resources=resources
))
elif subtype is LITERAL_IMAGE and 'Width' in xobj and 'Height' in xobj:
stack.complete_layout_items.append(make_image(
xobj, stack.t_matrix
))
else:
# unsupported xobject type.
pass
return stack
File renamed without changes.
@@ -0,0 +1,16 @@
from typing import List
from .PDFColorSpace import PDFColorSpace
class PDFColor:
def __init__(self, color_space: PDFColorSpace, *values: List[float]):
self.color_space: PDFColorSpace = color_space
self.values: list = values
def copy(self):
return self.__class__(
self.color_space,
*self.values
)
def __repr__(self):
return f"<PDFColor space={self.color_space} values={self.values}/>"
@@ -0,0 +1,27 @@
from collections import OrderedDict
class PDFColorSpace(object):
def __init__(self, name: str, ncomponents: int):
self.name: str = name
self.ncomponents: int = ncomponents
return
def __repr__(self):
return '<PDFColorSpace: %s, ncomponents=%d>' % (self.name, self.ncomponents)
PREDEFINED_COLORSPACE = OrderedDict()
for (name, n) in [
('DeviceGray', 1), # default value first
('CalRGB', 3),
('CalGray', 1),
('Lab', 3),
('DeviceRGB', 3),
('DeviceCMYK', 4),
('Separation', 1),
('Indexed', 1),
('Pattern', 1),
]:
PREDEFINED_COLORSPACE[name]=PDFColorSpace(name, n)
@@ -0,0 +1,55 @@
from .PDFColor import PDFColor, PDFColorSpace
from .PDFColorSpace import PREDEFINED_COLORSPACE
class PDFGraphicState(object):
class InvalidOperation(Exception): pass
def __init__(self):
self.linewidth = 0
self.linecap = None
self.linejoin = None
self.miterlimit = None
self.dash = None
self.intent = None
self.flatness = None
self.scolor:PDFColor = PDFColor(None)
self.ncolor:PDFColor = PDFColor(None)
self.scolspace:PDFColorSpace = None
self.ncolspace:PDFColorSpace = None
def set_stroke_color(self, colspace: PDFColorSpace, *values):
if colspace is None:
colspace = self.scolspace
self.scolor = PDFColor(colspace, *values)
def set_nostroke_color(self, colspace: PDFColorSpace, *values):
if colspace is None:
colspace = self.ncolspace
self.ncolor = PDFColor(colspace, *values)
def copy(self):
obj = PDFGraphicState()
obj.linewidth = self.linewidth
obj.linecap = self.linecap
obj.linejoin = self.linejoin
obj.miterlimit = self.miterlimit
obj.dash = self.dash
obj.intent = self.intent
obj.flatness = self.flatness
obj.scolor = self.scolor.copy()
obj.ncolor = self.ncolor.copy()
obj.scolspace = self.scolspace
obj.ncolspace = self.ncolspace
return obj
def __repr__(self):
return ('<PDFGraphicState linewidth=%r, linecap=%r, linejoin=%r, '
' miterlimit=%r, dash=%r, intent=%r, flatness=%r, '
' stroking color=%r, non stroking color=%r'
'>' %
(
self.linewidth, self.linecap, self.linejoin,
self.miterlimit, self.dash, self.intent, self.flatness,
str(self.scolor), str(self.ncolor)
)
)
@@ -0,0 +1,6 @@
from pdfmajor.parser.PDFStream import PDFStream
from ._base import PDFItem
class PDFImage(PDFItem):
def __init__(self, stream: PDFStream):
self.stream = stream
@@ -0,0 +1,12 @@
from typing import List
from ..PDFGraphicState import PDFGraphicState, PDFColorSpace
from ..Curves import CurvePath
from ._base import PDFItem
class PDFShape(PDFItem):
def __init__(self, gstate: PDFGraphicState, evenodd: bool, paths: List[CurvePath]):
self.gstate = gstate
self.evenodd = evenodd
self.paths = paths
@@ -1,96 +1,51 @@
# -*- coding: utf-8 -*-
from typing import List
from .PDFFont import PDFUnicodeNotDefined
from pdfmajor.utils import mult_matrix, isnumber, apply_matrix_pt, translate_matrix
from pdfmajor.utils import Bbox
from ..utils import isnumber, apply_matrix_pt
from ..utils import Bbox, mult_matrix, translate_matrix
from ..PDFTextState import PDFTextState
from ..PDFGraphicState import PDFGraphicState, PDFColorSpace
from ._base import PDFItem
from .PDFResourceManager import PDFResourceManager
from .PDFGraphicState import PDFGraphicState
from .PDFTextState import PDFTextState
from .PDFColorSpace import PDFColorSpace
from .PDFFont import PDFFont
## PDFDevice
##
class PDFDevice(object):
def __init__(self, rsrcmgr: PDFResourceManager):
self.rsrcmgr = rsrcmgr
self.ctm = None
return
class Char:
def __init__(self, char: str, bbox: Bbox):
self.char = char
self.bbox = bbox
def __str__(self):
return '"%s" %s' % (self.char, self.bbox)
def __repr__(self):
return '<PDFDevice>'
return f"<Char:{str(self)}/>"
def __enter__(self):
return self
class PDFText(PDFItem):
def __init__(self, chars: List[Char], textstate: PDFTextState, graphicstate: PDFGraphicState):
self.chars = chars
self.tstate = textstate
self.gstate = graphicstate
def __exit__(self, exc_type, exc_val, exc_tb):
self.close()
def close(self):
return
def set_ctm(self, ctm):
self.ctm = ctm
return
def begin_tag(self, tag, props=None):
return
def end_tag(self):
return
def do_tag(self, tag, props=None):
return
def begin_page(self, page, ctm):
return
def end_page(self, page):
return
def begin_figure(self, name, bbox: Bbox, matrix):
return
def end_figure(self, name):
return
def paint_path(self, graphicstate: PDFGraphicState, stroke, fill, evenodd, path):
return
def render_image(self, name, stream):
return
def render_string(self, textstate, seq, ncs, graphicstate: PDFGraphicState):
return
## PDFTextDevice
##
class PDFTextDevice(PDFDevice):
def render_string(self, textstate: PDFTextState, seq: bytearray, ncs: PDFColorSpace, graphicstate: PDFGraphicState):
textstate.matrix = mult_matrix(textstate.matrix, self.ctm)
@classmethod
def make(cls, seq: bytearray, ctm, textstate: PDFTextState, graphicstate: PDFGraphicState):
textstate.matrix = mult_matrix(textstate.matrix, ctm)
textstate.scaling = textstate.scaling * .01
textstate.charspace = textstate.charspace * textstate.scaling
textstate.wordspace = textstate.wordspace * textstate.scaling
ncs = graphicstate.ncolor.color_space
if textstate.font.is_multibyte():
textstate.wordspace = 0
dxscale = .001 * textstate.fontsize * textstate.scaling
if textstate.font.is_vertical():
self.__render_string_along(1,
return cls.__render_string_along(1,
seq, textstate, dxscale, ncs, graphicstate
)
else:
self.__render_string_along(0,
return cls.__render_string_along(0,
seq, textstate, dxscale, ncs, graphicstate
)
def __render_string_along(self, idx: int, seq: bytearray,
@classmethod
def __render_string_along(cls, idx: int, seq: bytearray,
textstate: PDFTextState, dxscale: float, ncs: PDFColorSpace, graphicstate: PDFGraphicState):
needcharspace = False
char_meta_datas = []
@@ -107,19 +62,20 @@ class PDFTextDevice(PDFDevice):
assert isinstance(text, str), str(type(text))
matrix = translate_matrix(textstate.matrix, textstate.linematrix)
adv, bbox = self.__compute_char_bbox(matrix, cid, textstate)
adv, bbox = cls.__compute_char_bbox(matrix, cid, textstate)
textstate.linematrix[idx] += adv
char_meta_datas.append(
( text, bbox, )
Char( text, Bbox.from_points(bbox), )
)
if cid == 32 and textstate.wordspace:
textstate.linematrix[idx] += textstate.wordspace
needcharspace = True
self.render_char_box(char_meta_datas, textstate, graphicstate, ncs)
return PDFText(char_meta_datas, textstate, graphicstate)
def __compute_char_bbox(self, matrix, char_id: int, textstate: PDFTextState):
@classmethod
def __compute_char_bbox(cls, matrix, char_id: int, textstate: PDFTextState):
font = textstate.font
fontsize = textstate.fontsize
rise = textstate.rise
@@ -157,7 +113,4 @@ class PDFTextDevice(PDFDevice):
if y1 < y0:
(y0, y1) = (y1, y0)
return (adv, ((x0, y0), (x1, y1)))
def render_char_box(self, chars: list, textstate: PDFTextState, graphicstate: PDFGraphicState, ncs: PDFColorSpace):
pass
return (adv, ((x0, y0), (x1, y1)))
@@ -0,0 +1,9 @@
from ._base import PDFItem
class PDFXObject(PDFItem):
def __init__(self, xobj_strem, bbox, resources, ctm):
self.stream = xobj_strem
self.bbox = bbox
self.resources = resources
self.ctm = ctm
self.children = []
@@ -0,0 +1,100 @@
from pdfmajor.utils import mult_matrix, isnumber, apply_matrix_pt, translate_matrix
from ..PDFTextState import PDFTextState
from ..PDFGraphicState import PDFGraphicState, PDFColorSpace
from .PDFItem import PDFText
from .Char import Char, Bbox
class TextDecoder:
@classmethod
def make_pdftext(cls, seq: bytearray, ctm, textstate: PDFTextState, graphicstate: PDFGraphicState) -> PDFText:
textstate.matrix = mult_matrix(textstate.matrix, ctm)
textstate.scaling = textstate.scaling * .01
textstate.charspace = textstate.charspace * textstate.scaling
textstate.wordspace = textstate.wordspace * textstate.scaling
ncs = graphicstate.ncolor.color_space
if textstate.font.is_multibyte():
textstate.wordspace = 0
dxscale = .001 * textstate.fontsize * textstate.scaling
if textstate.font.is_vertical():
return cls.__render_string_along(1,
seq, textstate, dxscale, ncs, graphicstate
)
else:
return cls.__render_string_along(0,
seq, textstate, dxscale, ncs, graphicstate
)
@classmethod
def __render_string_along(cls, idx: int, seq: bytearray,
textstate: PDFTextState, dxscale: float, ncs: PDFColorSpace, graphicstate: PDFGraphicState) -> PDFText:
needcharspace = False
char_meta_datas = []
for obj in seq:
if isnumber(obj):
textstate.linematrix[idx] -= obj*dxscale
needcharspace = True
else:
for cid in textstate.font.decode(obj):
if needcharspace:
textstate.linematrix[idx] += textstate.charspace
text = textstate.font.to_unichr(cid)
assert isinstance(text, str), str(type(text))
matrix = translate_matrix(textstate.matrix, textstate.linematrix)
adv, bbox = cls.__compute_char_bbox(matrix, cid, textstate)
textstate.linematrix[idx] += adv
char_meta_datas.append(
Char( text, Bbox.from_points(bbox), )
)
if cid == 32 and textstate.wordspace:
textstate.linematrix[idx] += textstate.wordspace
needcharspace = True
return PDFText(char_meta_datas, textstate, graphicstate)
@classmethod
def __compute_char_bbox(cls, matrix, char_id: int, textstate: PDFTextState):
font = textstate.font
fontsize = textstate.fontsize
rise = textstate.rise
scaling = textstate.scaling
textwidth = font.char_width(char_id)
textdisp = font.char_disp(char_id)
adv = textwidth * fontsize * scaling
# compute the boundary rectangle.
if font.is_vertical():
# vertical
width = font.get_width() * fontsize
(vx, vy) = textdisp
if vx is None:
vx = width * 0.5
else:
vx = vx * fontsize * .001
vy = (1000 - vy) * fontsize * .001
tx = -vx
ty = vy + rise
bll = (tx, ty+adv)
bur = (tx+width, ty)
else:
# horizontal
height = font.get_height() * fontsize
descent = font.get_descent() * fontsize
ty = descent + rise
bll = (0, ty)
bur = (adv, ty+height)
(x0, y0) = apply_matrix_pt(matrix, bll)
(x1, y1) = apply_matrix_pt(matrix, bur)
if x1 < x0:
(x0, x1) = (x1, x0)
if y1 < y0:
(y0, y1) = (y1, y0)
return (adv, ((x0, y0), (x1, y1)))
@@ -0,0 +1,5 @@
from ._base import PDFItem
from .PDFShape import PDFShape
from .PDFImage import PDFImage
from .PDFText import PDFText
from .PDFXObject import PDFXObject
@@ -0,0 +1,6 @@
class PDFItem:
def __repr__(self) -> str:
return f"""<{self.__class__.__name__} {' '.join([
f'{name}="{val}"'
for name, val in vars(self).items()
])}/>"""
@@ -0,0 +1,246 @@
## CFFFont
## (Format specified in Adobe Technical Note: #5176
## "The Compact Font Format Specification")
##
import struct
from io import BytesIO
from pdfmajor.utils import nunpack
NIBBLES = ('0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '.', 'e', 'e-', None, '-')
def getdict(data):
d = {}
fp = BytesIO(data)
stack = []
while 1:
c = fp.read(1)
if not c:
break
b0 = ord(c)
if b0 <= 21:
d[b0] = stack
stack = []
continue
if b0 == 30:
s = ''
loop = True
while loop:
b = ord(fp.read(1))
for n in (b >> 4, b & 15):
if n == 15:
loop = False
else:
s += NIBBLES[n]
value = float(s)
elif 32 <= b0 and b0 <= 246:
value = b0-139
else:
b1 = ord(fp.read(1))
if 247 <= b0 and b0 <= 250:
value = ((b0-247) << 8)+b1+108
elif 251 <= b0 and b0 <= 254:
value = -((b0-251) << 8)-b1-108
else:
b2 = ord(fp.read(1))
if 128 <= b1:
b1 -= 256
if b0 == 28:
value = b1 << 8 | b2
else:
value = b1 << 24 | b2 << 16 | struct.unpack('>H', fp.read(2))[0]
stack.append(value)
return d
class CFFFont(object):
STANDARD_STRINGS = (
'.notdef', 'space', 'exclam', 'quotedbl', 'numbersign',
'dollar', 'percent', 'ampersand', 'quoteright', 'parenleft',
'parenright', 'asterisk', 'plus', 'comma', 'hyphen', 'period',
'slash', 'zero', 'one', 'two', 'three', 'four', 'five', 'six',
'seven', 'eight', 'nine', 'colon', 'semicolon', 'less', 'equal',
'greater', 'question', 'at', 'A', 'B', 'C', 'D', 'E', 'F', 'G',
'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T',
'U', 'V', 'W', 'X', 'Y', 'Z', 'bracketleft', 'backslash',
'bracketright', 'asciicircum', 'underscore', 'quoteleft', 'a',
'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n',
'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z',
'braceleft', 'bar', 'braceright', 'asciitilde', 'exclamdown',
'cent', 'sterling', 'fraction', 'yen', 'florin', 'section',
'currency', 'quotesingle', 'quotedblleft', 'guillemotleft',
'guilsinglleft', 'guilsinglright', 'fi', 'fl', 'endash',
'dagger', 'daggerdbl', 'periodcentered', 'paragraph', 'bullet',
'quotesinglbase', 'quotedblbase', 'quotedblright',
'guillemotright', 'ellipsis', 'perthousand', 'questiondown',
'grave', 'acute', 'circumflex', 'tilde', 'macron', 'breve',
'dotaccent', 'dieresis', 'ring', 'cedilla', 'hungarumlaut',
'ogonek', 'caron', 'emdash', 'AE', 'ordfeminine', 'Lslash',
'Oslash', 'OE', 'ordmasculine', 'ae', 'dotlessi', 'lslash',
'oslash', 'oe', 'germandbls', 'onesuperior', 'logicalnot', 'mu',
'trademark', 'Eth', 'onehalf', 'plusminus', 'Thorn',
'onequarter', 'divide', 'brokenbar', 'degree', 'thorn',
'threequarters', 'twosuperior', 'registered', 'minus', 'eth',
'multiply', 'threesuperior', 'copyright', 'Aacute',
'Acircumflex', 'Adieresis', 'Agrave', 'Aring', 'Atilde',
'Ccedilla', 'Eacute', 'Ecircumflex', 'Edieresis', 'Egrave',
'Iacute', 'Icircumflex', 'Idieresis', 'Igrave', 'Ntilde',
'Oacute', 'Ocircumflex', 'Odieresis', 'Ograve', 'Otilde',
'Scaron', 'Uacute', 'Ucircumflex', 'Udieresis', 'Ugrave',
'Yacute', 'Ydieresis', 'Zcaron', 'aacute', 'acircumflex',
'adieresis', 'agrave', 'aring', 'atilde', 'ccedilla', 'eacute',
'ecircumflex', 'edieresis', 'egrave', 'iacute', 'icircumflex',
'idieresis', 'igrave', 'ntilde', 'oacute', 'ocircumflex',
'odieresis', 'ograve', 'otilde', 'scaron', 'uacute',
'ucircumflex', 'udieresis', 'ugrave', 'yacute', 'ydieresis',
'zcaron', 'exclamsmall', 'Hungarumlautsmall', 'dollaroldstyle',
'dollarsuperior', 'ampersandsmall', 'Acutesmall',
'parenleftsuperior', 'parenrightsuperior', 'twodotenleader',
'onedotenleader', 'zerooldstyle', 'oneoldstyle', 'twooldstyle',
'threeoldstyle', 'fouroldstyle', 'fiveoldstyle', 'sixoldstyle',
'sevenoldstyle', 'eightoldstyle', 'nineoldstyle',
'commasuperior', 'threequartersemdash', 'periodsuperior',
'questionsmall', 'asuperior', 'bsuperior', 'centsuperior',
'dsuperior', 'esuperior', 'isuperior', 'lsuperior', 'msuperior',
'nsuperior', 'osuperior', 'rsuperior', 'ssuperior', 'tsuperior',
'ff', 'ffi', 'ffl', 'parenleftinferior', 'parenrightinferior',
'Circumflexsmall', 'hyphensuperior', 'Gravesmall', 'Asmall',
'Bsmall', 'Csmall', 'Dsmall', 'Esmall', 'Fsmall', 'Gsmall',
'Hsmall', 'Ismall', 'Jsmall', 'Ksmall', 'Lsmall', 'Msmall',
'Nsmall', 'Osmall', 'Psmall', 'Qsmall', 'Rsmall', 'Ssmall',
'Tsmall', 'Usmall', 'Vsmall', 'Wsmall', 'Xsmall', 'Ysmall',
'Zsmall', 'colonmonetary', 'onefitted', 'rupiah', 'Tildesmall',
'exclamdownsmall', 'centoldstyle', 'Lslashsmall', 'Scaronsmall',
'Zcaronsmall', 'Dieresissmall', 'Brevesmall', 'Caronsmall',
'Dotaccentsmall', 'Macronsmall', 'figuredash', 'hypheninferior',
'Ogoneksmall', 'Ringsmall', 'Cedillasmall', 'questiondownsmall',
'oneeighth', 'threeeighths', 'fiveeighths', 'seveneighths',
'onethird', 'twothirds', 'zerosuperior', 'foursuperior',
'fivesuperior', 'sixsuperior', 'sevensuperior', 'eightsuperior',
'ninesuperior', 'zeroinferior', 'oneinferior', 'twoinferior',
'threeinferior', 'fourinferior', 'fiveinferior', 'sixinferior',
'seveninferior', 'eightinferior', 'nineinferior',
'centinferior', 'dollarinferior', 'periodinferior',
'commainferior', 'Agravesmall', 'Aacutesmall',
'Acircumflexsmall', 'Atildesmall', 'Adieresissmall',
'Aringsmall', 'AEsmall', 'Ccedillasmall', 'Egravesmall',
'Eacutesmall', 'Ecircumflexsmall', 'Edieresissmall',
'Igravesmall', 'Iacutesmall', 'Icircumflexsmall',
'Idieresissmall', 'Ethsmall', 'Ntildesmall', 'Ogravesmall',
'Oacutesmall', 'Ocircumflexsmall', 'Otildesmall',
'Odieresissmall', 'OEsmall', 'Oslashsmall', 'Ugravesmall',
'Uacutesmall', 'Ucircumflexsmall', 'Udieresissmall',
'Yacutesmall', 'Thornsmall', 'Ydieresissmall', '001.000',
'001.001', '001.002', '001.003', 'Black', 'Bold', 'Book',
'Light', 'Medium', 'Regular', 'Roman', 'Semibold',
)
class INDEX(object):
def __init__(self, fp):
self.fp = fp
self.offsets = []
(count, offsize) = struct.unpack('>HB', self.fp.read(3))
for _ in range(count+1):
self.offsets.append(nunpack(self.fp.read(offsize)))
self.base = self.fp.tell()-1
self.fp.seek(self.base+self.offsets[-1])
return
def __repr__(self):
return '<INDEX: size=%d>' % len(self)
def __len__(self): # pylint: disable=E0303
return len(self.offsets)-1
def __getitem__(self, i):
self.fp.seek(self.base+self.offsets[i])
return self.fp.read(self.offsets[i+1]-self.offsets[i])
def __iter__(self):
return iter(self[i] for i in range(len(self)))
def __init__(self, name, fp):
self.name = name
self.fp = fp
# Header
(_major, _minor, hdrsize, _) = struct.unpack('BBBB', self.fp.read(4))
self.fp.read(hdrsize-4)
# Name INDEX
self.name_index = self.INDEX(self.fp)
# Top DICT INDEX
self.dict_index = self.INDEX(self.fp)
# String INDEX
self.string_index = self.INDEX(self.fp)
# Global Subr INDEX
self.subr_index = self.INDEX(self.fp)
# Top DICT DATA
self.top_dict = getdict(self.dict_index[0])
(charset_pos,) = self.top_dict.get(15, [0])
(encoding_pos,) = self.top_dict.get(16, [0])
(charstring_pos,) = self.top_dict.get(17, [0])
# CharStrings
self.fp.seek(charstring_pos)
self.charstring = self.INDEX(self.fp)
self.nglyphs = len(self.charstring)
# Encodings
self.code2gid = {}
self.gid2code = {}
self.fp.seek(encoding_pos)
format = self.fp.read(1)
if format == b'\x00':
# Format 0
(n,) = struct.unpack('B', self.fp.read(1))
for (code, gid) in enumerate(struct.unpack('B'*n, self.fp.read(n))):
self.code2gid[code] = gid
self.gid2code[gid] = code
elif format == b'\x01':
# Format 1
(n,) = struct.unpack('B', self.fp.read(1))
code = 0
for _ in range(n):
(first, nleft) = struct.unpack('BB', self.fp.read(2))
for gid in range(first, first+nleft+1):
self.code2gid[code] = gid
self.gid2code[gid] = code
code += 1
else:
raise ValueError('unsupported encoding format: %r' % format)
# Charsets
self.name2gid = {}
self.gid2name = {}
self.fp.seek(charset_pos)
format = self.fp.read(1)
if format == b'\x00':
# Format 0
n = self.nglyphs-1
for (gid, sid) in enumerate(struct.unpack('>'+'H'*n, self.fp.read(2*n))):
gid += 1
name = self.getstr(sid)
self.name2gid[name] = gid
self.gid2name[gid] = name
elif format == b'\x01':
# Format 1
(n,) = struct.unpack('B', self.fp.read(1))
sid = 0
for _ in range(n):
(first, nleft) = struct.unpack('BB', self.fp.read(2))
for gid in range(first, first+nleft+1):
name = self.getstr(sid)
self.name2gid[name] = gid
self.gid2name[gid] = name
sid += 1
elif format == b'\x02':
# Format 2
assert False, str(('Unhandled', format))
else:
raise ValueError('unsupported charset format: %r' % format)
#print self.code2gid
#print self.name2gid
#assert 0
return
def getstr(self, sid):
if sid < len(self.STANDARD_STRINGS):
return self.STANDARD_STRINGS[sid]
return self.string_index[sid-len(self.STANDARD_STRINGS)]
@@ -0,0 +1,140 @@
# PDFFont
from abc import abstractmethod
from io import BytesIO
from pdfmajor.parser.EncodingDB import EncodingDB
from pdfmajor.parser.PSStackParser import LIT
from pdfmajor.parser.PSStackParser import PSLiteral
from pdfmajor.parser.PSStackParser import literal_name
from pdfmajor.parser.PDFStream import resolve1
from pdfmajor.parser.PDFStream import int_value
from pdfmajor.parser.PDFStream import num_value
from pdfmajor.parser.PDFStream import list_value
from pdfmajor.parser.PDFStream import PDFException
from pdfmajor.parser.PDFStream import PDFStream
from pdfmajor.parser.cmapdb import FileUnicodeMap
from pdfmajor.parser.cmapdb import CMapParser
## Literals
##
LITERAL_STANDARD_ENCODING = LIT('StandardEncoding')
LITERAL_TYPE1C = LIT('Type1C')
class PDFFont(object):
class PDFFontError(PDFException): pass
class PDFUnicodeNotDefined(PDFFontError): pass
def __init__(self, descriptor, widths, default_width=None):
self.descriptor = descriptor
self.widths = widths
self.fontname = resolve1(descriptor.get('FontName', 'unknown'))
if isinstance(self.fontname, PSLiteral):
self.fontname = literal_name(self.fontname)
self.flags = int_value(descriptor.get('Flags', 0))
self.ascent = num_value(descriptor.get('Ascent', 0))
self.descent = num_value(descriptor.get('Descent', 0))
self.italic_angle = num_value(descriptor.get('ItalicAngle', 0))
self.default_width = default_width or num_value(descriptor.get('MissingWidth', 0))
self.leading = num_value(descriptor.get('Leading', 0))
self.bbox = list_value(descriptor.get('FontBBox', (0, 0, 0, 0)))
self.hscale = self.vscale = .001
return
def __repr__(self):
return '<PDFFont>'
def is_vertical(self):
return False
def is_multibyte(self):
return False
def decode(self, bytes):
return bytearray(bytes) # map(ord, bytes)
def get_ascent(self):
return self.ascent * self.vscale
def get_descent(self):
return self.descent * self.vscale
def get_width(self):
w = self.bbox[2]-self.bbox[0]
if w == 0:
w = -self.default_width
return w * self.hscale
def get_height(self):
h = self.bbox[3]-self.bbox[1]
if h == 0:
h = self.ascent - self.descent
return h * self.vscale
def char_width(self, cid):
try:
return self.widths[cid] * self.hscale
except KeyError:
try:
return self.widths[self.to_unichr(cid)] * self.hscale
except (KeyError, self.PDFUnicodeNotDefined):
return self.default_width * self.hscale
def char_disp(self, cid):
return 0
def string_width(self, s):
return sum(self.char_width(cid) for cid in self.decode(s))
@abstractmethod
def to_unichr(self, cid):
raise NotImplementedError
@property
def font_weight(self):
if 'FontWeight' in self.descriptor.keys():
return self.descriptor['FontWeight']
elif 'bold' in self.fontname.lower():
return 'bold'
else:
return None
@property
def is_bold(self):
return self.font_weight == 'bold' or self.font_weight > 549
# PDFSimpleFont
class PDFSimpleFont(PDFFont):
def __init__(self, descriptor, widths, spec):
# Font encoding is specified either by a name of
# built-in encoding or a dictionary that describes
# the differences.
if 'Encoding' in spec:
encoding = resolve1(spec['Encoding'])
else:
encoding = LITERAL_STANDARD_ENCODING
if isinstance(encoding, dict):
name = literal_name(encoding.get('BaseEncoding', LITERAL_STANDARD_ENCODING))
diff = list_value(encoding.get('Differences', []))
self.cid2unicode = EncodingDB.get_encoding(name, diff)
else:
self.cid2unicode = EncodingDB.get_encoding(literal_name(encoding))
self.unicode_map = None
if 'ToUnicode' in spec:
strm = PDFStream.validated_stream(spec['ToUnicode'])
self.unicode_map = FileUnicodeMap()
CMapParser(self.unicode_map, BytesIO(strm.get_data())).run()
PDFFont.__init__(self, descriptor, widths)
return
def to_unichr(self, cid):
if self.unicode_map:
try:
return self.unicode_map.get_unichr(cid)
except KeyError:
pass
try:
return self.cid2unicode[cid]
except KeyError:
raise self.PDFUnicodeNotDefined(None, cid)
@@ -0,0 +1,89 @@
## TrueTypeFont
##
import struct
from pdfmajor.parser.cmapdb import FileUnicodeMap
class TrueTypeFont(object):
class CMapNotFound(Exception):
pass
def __init__(self, name, fp):
self.name = name
self.fp = fp
self.tables = {}
self.fonttype = fp.read(4)
try:
(ntables, _1, _2, _3) = struct.unpack('>HHHH', fp.read(8))
for _ in range(ntables):
(name, _, offset, length) = struct.unpack('>4sLLL', fp.read(16))
self.tables[name] = (offset, length)
except struct.error:
# Do not fail if there are not enough bytes to read. Even for
# corrupted PDFs we would like to get as much information as
# possible, so continue.
pass
return
def create_unicode_map(self):
if 'cmap' not in self.tables:
raise TrueTypeFont.CMapNotFound
(base_offset, _) = self.tables['cmap']
fp = self.fp
fp.seek(base_offset)
(_, nsubtables) = struct.unpack('>HH', fp.read(4))
subtables = []
for i in range(nsubtables):
subtables.append(struct.unpack('>HHL', fp.read(8)))
char2gid = {}
# Only supports subtable type 0, 2 and 4.
for (_1, _2, st_offset) in subtables:
fp.seek(base_offset+st_offset)
(fmttype, _, _) = struct.unpack('>HHH', fp.read(6))
if fmttype == 0:
char2gid.update(enumerate(struct.unpack('>256B', fp.read(256))))
elif fmttype == 2:
subheaderkeys = struct.unpack('>256H', fp.read(512))
firstbytes = [0]*8192
for (i, k) in enumerate(subheaderkeys):
firstbytes[k//8] = i
nhdrs = max(subheaderkeys)//8 + 1
hdrs = []
for i in range(nhdrs):
(firstcode, entcount, delta, offset) = struct.unpack('>HHhH', fp.read(8))
hdrs.append((i, firstcode, entcount, delta, fp.tell()-2+offset))
for (i, firstcode, entcount, delta, pos) in hdrs:
if not entcount:
continue
first = firstcode + (firstbytes[i] << 8)
fp.seek(pos)
for c in range(entcount):
gid = struct.unpack('>H', fp.read(2))
if gid:
gid += delta
char2gid[first+c] = gid
elif fmttype == 4:
(segcount, _1, _2, _3) = struct.unpack('>HHHH', fp.read(8))
segcount //= 2
ecs = struct.unpack('>%dH' % segcount, fp.read(2*segcount))
fp.read(2)
scs = struct.unpack('>%dH' % segcount, fp.read(2*segcount))
idds = struct.unpack('>%dh' % segcount, fp.read(2*segcount))
pos = fp.tell()
idrs = struct.unpack('>%dH' % segcount, fp.read(2*segcount))
for (ec, sc, idd, idr) in zip(ecs, scs, idds, idrs):
if idr:
fp.seek(pos+idr)
for c in range(sc, ec+1):
char2gid[c] = (struct.unpack('>H', fp.read(2))[0] + idd) & 0xffff
else:
for c in range(sc, ec+1):
char2gid[c] = (c + idd) & 0xffff
else:
assert False, str(('Unhandled', fmttype))
# create unicode map
unicode_map = FileUnicodeMap()
for (char, gid) in char2gid.items():
unicode_map.add_cid2unichr(gid, char)
return unicode_map
@@ -0,0 +1,45 @@
from pdfmajor.parser.PSStackParser import PSStackParser
from pdfmajor.parser.PSStackParser import PSEOF, KWD
from pdfmajor.parser.PSStackParser import PSLiteral
from pdfmajor.parser.PSStackParser import literal_name
from pdfmajor.parser.EncodingDB import name2unicode
## Type1FontHeaderParser
##
class Type1FontHeaderParser(PSStackParser):
KEYWORD_BEGIN = KWD(b'begin')
KEYWORD_END = KWD(b'end')
KEYWORD_DEF = KWD(b'def')
KEYWORD_PUT = KWD(b'put')
KEYWORD_DICT = KWD(b'dict')
KEYWORD_ARRAY = KWD(b'array')
KEYWORD_READONLY = KWD(b'readonly')
KEYWORD_FOR = KWD(b'for')
KEYWORD_FOR = KWD(b'for')
def __init__(self, data):
PSStackParser.__init__(self, data)
self._cid2unicode = {}
return
def get_encoding(self):
while 1:
try:
(cid, name) = self.nextobject()
except PSEOF:
break
try:
self._cid2unicode[cid] = name2unicode(name)
except KeyError:
pass
return self._cid2unicode
def do_keyword(self, pos, token):
if token is self.KEYWORD_PUT:
((_, key), (_, value)) = self.pop(2) # pylint: disable=E0632
if (isinstance(key, int) and
isinstance(value, PSLiteral)):
self.add_results((key, literal_name(value)))
return
@@ -0,0 +1,57 @@
from pdfmajor.utils import get_logger, settings
from pdfmajor.parser.PSStackParser import literal_name
from pdfmajor.parser.PDFStream import list_value
from pdfmajor.parser.PDFStream import dict_value
from pdfmajor.parser.PDFStream import resolve1
from pdfmajor.parser.constants import LITERAL_FONT
from .fonts import PDFType1Font, PDFTrueTypeFont, PDFType3Font, PDFCIDFont, PDFFont
log = get_logger(__file__)
def get_font(objid: int, spec: dict, cached_fonts: dict = {}):
if objid and objid in cached_fonts:
font = cached_fonts[objid]
else:
log.debug('get_font: create: objid=%r, spec=%r', objid, spec)
if settings.STRICT:
if spec['Type'] is not LITERAL_FONT:
raise PDFFont.PDFFontError('Type is not /Font')
# Create a Font object.
if 'Subtype' in spec:
subtype = literal_name(spec['Subtype'])
else:
if settings.STRICT:
raise PDFFont.PDFFontError('Font Subtype is not specified.')
subtype = 'Type1'
if subtype in ('Type1', 'MMType1'):
# Type1 Font
font = PDFType1Font(spec)
elif subtype == 'TrueType':
# TrueType Font
font = PDFTrueTypeFont(spec)
elif subtype == 'Type3':
# Type3 Font
font = PDFType3Font(spec)
elif subtype in ('CIDFontType0', 'CIDFontType2'):
# CID Font
font = PDFCIDFont(spec)
elif subtype == 'Type0':
# Type0 Font
dfonts = list_value(spec['DescendantFonts'])
assert dfonts
subspec = dict_value(dfonts[0]).copy()
for k in ('Encoding', 'ToUnicode'):
if k in spec:
subspec[k] = resolve1(spec[k])
font = get_font(None, subspec, cached_fonts)
else:
if settings.STRICT:
raise PDFFont.PDFFontError('Invalid Font spec: %r' % spec)
font = PDFType1Font(spec) # this is so wrong!
if objid:
cached_fonts[objid] = font
return font
@@ -0,0 +1,174 @@
from io import BytesIO
from pdfmajor.parser.PSStackParser import literal_name
from pdfmajor.parser.PDFStream import int_value
from pdfmajor.parser.PDFStream import num_value
from pdfmajor.parser.PDFStream import list_value
from pdfmajor.parser.PDFStream import dict_value
from pdfmajor.parser.PDFStream import PDFStream
from pdfmajor.parser.PDFStream import resolve1
from pdfmajor.parser.cmapdb import CMap, CMapDB, CMapParser
from pdfmajor.parser.cmapdb import FileUnicodeMap
from pdfmajor.utils import settings, apply_matrix_norm
from .PDFFont import PDFFont, PDFSimpleFont
from .util import FontMetricsDB, get_widths, get_widths2
from .Type1FontHeaderParser import Type1FontHeaderParser
from .TrueTypeFont import TrueTypeFont
# PDFType1Font
class PDFType1Font(PDFSimpleFont):
def __init__(self, spec):
try:
self.basefont = literal_name(spec['BaseFont'])
except KeyError:
if settings.STRICT:
raise self.PDFFontError('BaseFont is missing')
self.basefont = 'unknown'
try:
(descriptor, widths) = FontMetricsDB.get_metrics(self.basefont)
except KeyError:
descriptor = dict_value(spec.get('FontDescriptor', {}))
firstchar = int_value(spec.get('FirstChar', 0))
#lastchar = int_value(spec.get('LastChar', 255))
widths = list_value(spec.get('Widths', [0]*256))
widths = dict((i+firstchar, w) for (i, w) in enumerate(widths))
PDFSimpleFont.__init__(self, descriptor, widths, spec)
if 'Encoding' not in spec and 'FontFile' in descriptor:
# try to recover the missing encoding info from the font file.
self.fontfile = PDFStream.validated_stream(descriptor.get('FontFile'))
length1 = int_value(self.fontfile['Length1'])
data = self.fontfile.get_data()[:length1]
parser = Type1FontHeaderParser(BytesIO(data))
self.cid2unicode = parser.get_encoding()
return
def __repr__(self):
return '<PDFType1Font: basefont=%r>' % self.basefont
# PDFTrueTypeFont
class PDFTrueTypeFont(PDFType1Font):
def __repr__(self):
return '<PDFTrueTypeFont: basefont=%r>' % self.basefont
# PDFType3Font
class PDFType3Font(PDFSimpleFont):
def __init__(self, spec):
firstchar = int_value(spec.get('FirstChar', 0))
#lastchar = int_value(spec.get('LastChar', 0))
widths = list_value(spec.get('Widths', [0]*256))
widths = dict((i+firstchar, w) for (i, w) in enumerate(widths))
if 'FontDescriptor' in spec:
descriptor = dict_value(spec['FontDescriptor'])
else:
descriptor = {'Ascent': 0, 'Descent': 0,
'FontBBox': spec['FontBBox']}
PDFSimpleFont.__init__(self, descriptor, widths, spec)
self.matrix = tuple(list_value(spec.get('FontMatrix')))
(_, self.descent, _, self.ascent) = self.bbox
(self.hscale, self.vscale) = apply_matrix_norm(self.matrix, (1, 1))
return
def __repr__(self):
return '<PDFType3Font>'
# PDFCIDFont
class PDFCIDFont(PDFFont):
def __init__(self, spec, strict=settings.STRICT):
try:
self.basefont = literal_name(spec['BaseFont'])
except KeyError:
if strict:
raise self.PDFFontError('BaseFont is missing')
self.basefont = 'unknown'
self.cidsysteminfo = dict_value(spec.get('CIDSystemInfo', {}))
self.cidcoding = '%s-%s' % (resolve1(self.cidsysteminfo.get('Registry', b'unknown')).decode("latin1"),
resolve1(self.cidsysteminfo.get('Ordering', b'unknown')).decode("latin1"))
try:
name = literal_name(spec['Encoding'])
except KeyError:
if strict:
raise self.PDFFontError('Encoding is unspecified')
name = 'unknown'
try:
self.cmap = CMapDB.get_cmap(name)
except CMapDB.CMapNotFound as e:
if strict:
raise self.PDFFontError(e)
self.cmap = CMap()
try:
descriptor = dict_value(spec['FontDescriptor'])
except KeyError:
if strict:
raise self.PDFFontError('FontDescriptor is missing')
descriptor = {}
ttf = None
if 'FontFile2' in descriptor:
self.fontfile = PDFStream.validated_stream(descriptor.get('FontFile2'))
ttf = TrueTypeFont(self.basefont,
BytesIO(self.fontfile.get_data()))
self.unicode_map = None
if 'ToUnicode' in spec:
strm = PDFStream.validated_stream(spec['ToUnicode'])
self.unicode_map = FileUnicodeMap()
CMapParser(self.unicode_map, BytesIO(strm.get_data())).run()
elif self.cidcoding in ('Adobe-Identity', 'Adobe-UCS'):
if ttf:
try:
self.unicode_map = ttf.create_unicode_map()
except TrueTypeFont.CMapNotFound:
pass
else:
try:
self.unicode_map = CMapDB.get_unicode_map(self.cidcoding, self.cmap.is_vertical())
except CMapDB.CMapNotFound as e:
pass
self.vertical = self.cmap.is_vertical()
if self.vertical:
# writing mode: vertical
widths = get_widths2(list_value(spec.get('W2', [])))
self.disps = dict((cid, (vx, vy)) for (cid, (_, (vx, vy))) in iter(widths.items()))
(vy, w) = spec.get('DW2', [880, -1000])
self.default_disp = (None, vy)
widths = dict((cid, w) for (cid, (w, _)) in iter(widths.items()))
default_width = w
else:
# writing mode: horizontal
self.disps = {}
self.default_disp = 0
widths = get_widths(list_value(spec.get('W', [])))
default_width = spec.get('DW', 1000)
PDFFont.__init__(self, descriptor, widths, default_width=default_width)
return
def __repr__(self):
return '<PDFCIDFont: basefont=%r, cidcoding=%r>' % (self.basefont, self.cidcoding)
def is_vertical(self):
return self.vertical
def is_multibyte(self):
return True
def decode(self, bytes):
return self.cmap.decode(bytes)
def char_disp(self, cid):
"Returns an integer for horizontal fonts, a tuple for vertical fonts."
return self.disps.get(cid, self.default_disp)
def to_unichr(self, cid):
try:
if not self.unicode_map:
raise KeyError(cid)
return self.unicode_map.get_unichr(cid)
except KeyError:
raise self.PDFUnicodeNotDefined(self.cidcoding, cid)
@@ -0,0 +1,57 @@
from pdfmajor.utils import FONT_METRICS, choplist, isnumber
def get_widths(seq):
widths = {}
r = []
for v in seq:
if isinstance(v, list):
if r:
char1 = r[-1]
for (i, w) in enumerate(v):
widths[char1+i] = w
r = []
elif isnumber(v):
r.append(v)
if len(r) == 3:
(char1, char2, w) = r
for i in range(char1, char2+1):
widths[i] = w
r = []
return widths
assert get_widths([1]) == {}
assert get_widths([1,2,3]) == {1:3, 2:3}
assert get_widths([1,[2,3],6,[7,8]]) == {1:2,2:3, 6:7,7:8}
def get_widths2(seq):
widths = {}
r = []
for v in seq:
if isinstance(v, list):
if r:
char1 = r[-1]
for (i, (w, vx, vy)) in enumerate(choplist(3, v)):
widths[char1+i] = (w, (vx, vy))
r = []
elif isnumber(v):
r.append(v)
if len(r) == 5:
(char1, char2, w, vx, vy) = r # pylint: disable=E0632
for i in range(char1, char2+1):
widths[i] = (w, (vx, vy))
r = []
return widths
assert get_widths2([1]) == {}
assert get_widths2([1,2,3,4,5]) == {1:(3, (4,5)), 2:(3, (4,5))}
assert get_widths2([1,[2,3,4,5],6,[7,8,9]]) == {1:(2, (3,4)), 6:(7, (8,9))}
## FontMetricsDB
##
class FontMetricsDB(object):
@classmethod
def get_metrics(klass, fontname):
return FONT_METRICS[fontname]
@@ -1,11 +1,10 @@
from .PDFFont import PDFFont
from ..utils import MATRIX_IDENTITY
from .PDFFont import PDFFont, get_font
from pdfmajor.utils import MATRIX_IDENTITY
## PDFTextState
##
class PDFTextState(object):
def __init__(self):
self.font: PDFFont = None
self.fontsize: int = 0
@@ -15,10 +14,8 @@ class PDFTextState(object):
self.leading: int = 0
self.render: int = 0
self.rise: int = 0
self.reset()
# self.matrix is set
# self.linematrix is set
return
self.matrix = MATRIX_IDENTITY
self.linematrix = [0, 0]
def __repr__(self):
return ('<PDFTextState: font=%r, fontsize=%r, charspace=%r, wordspace=%r, '
@@ -29,7 +26,7 @@ class PDFTextState(object):
self.matrix, self.linematrix))
def copy(self):
obj = PDFTextState()
obj = self.__class__()
obj.font = self.font
obj.fontsize = self.fontsize
obj.charspace = self.charspace
@@ -42,12 +39,10 @@ class PDFTextState(object):
obj.linematrix = self.linematrix
return obj
def reset(self):
def reset_matrices(self):
self.matrix = MATRIX_IDENTITY
self.linematrix = [0, 0]
return
@property
def is_upright(self):
(a, b, c, d, _, _) = self.matrix
return (0 < a*d*self.scaling and b*c <= 0)
return (0 < self.matrix[:,0].prod()*self.scaling and self.matrix[0,1:].prod() <= 0)
@@ -0,0 +1,39 @@
from typing import List
from pdfmajor.utils import MATRIX_IDENTITY
from .PDFGraphicState import PDFGraphicState
from .PDFGraphicState.PDFColor import PDFColor
from .PDFGraphicState.PDFColorSpace import PREDEFINED_COLORSPACE, OrderedDict, PDFColorSpace
from .PDFTextState import PDFTextState, PDFFont, get_font
from .Curves import CurvePath
from .layout import LTItem, LTComponent, LTContainer
from .layout import LTCharBlock, LTChar
from .layout import LTCurve, LTLine, LTHorizontalLine, LTVerticalLine, LTRect
from .layout import LTXObject
from .layout import LTImage
from .layout import make_char_block, make_curve, make_image, make_xobject
class PDFStateStack:
def __init__(self):
self.t_matrix = MATRIX_IDENTITY
self.text = PDFTextState()
self.graphics = PDFGraphicState()
self.gstack : list = []
self.curvestacks: List[CurvePath] = []
self.argstack: List[bytes] = []
self.complete_layout_items : List[LTItem] = []
self.colorspace_map: OrderedDict = PREDEFINED_COLORSPACE.copy()
self.fontmap = {}
self.xobjmap = {}
self.resources = {}
def pop(self, n: int) -> bytearray:
if n == 0:
return []
x = self.argstack[-n:]
self.argstack = self.argstack[:-n]
return x
@@ -1,9 +1,7 @@
from ._base import LTComponent
from ..utils import apply_matrix_norm, apply_matrix_pt, matrix2str, Bbox
from ..interpreter.PDFGraphicState import PDFGraphicState
from ..interpreter.PDFFont import PDFFont
from ..interpreter.PDFColorSpace import PDFColorSpace
from ..interpreter.PDFTextState import PDFTextState
from pdfmajor.utils import apply_matrix_norm, apply_matrix_pt, matrix2str, Bbox
from ..PDFGraphicState import PDFColor
from ..PDFTextState import PDFTextState, PDFFont
## LTChar
##
@@ -13,16 +11,13 @@ class LTChar(LTComponent):
bbox: Bbox,
char: str,
textstate: PDFTextState,
ncs: PDFColorSpace,
graphicstate: PDFGraphicState
color: PDFColor
):
self._text = char
self.font = textstate.font
self.textstate = textstate
self.ncs = ncs
self.graphicstate = graphicstate
self.color = color
LTComponent.__init__(self, bbox)
if self.font.is_vertical():
if self.textstate.font.is_vertical():
self.size = self.width
else:
self.size = self.height
@@ -42,6 +37,10 @@ class LTChar(LTComponent):
def get_text(self):
return self._text
@property
def font(self):
return self.textstate.font
@property
def fontname(self) -> str:
return self.font.fontname
@@ -3,55 +3,49 @@ from typing import List
from ._base import LTContainer
from .LTChar import LTChar
from ..utils import apply_matrix_norm, apply_matrix_pt, matrix2str, Bbox, Point
from ..interpreter.PDFGraphicState import PDFGraphicState
from ..interpreter.PDFFont import PDFFont
from ..interpreter.PDFColorSpace import PDFColorSpace
from ..interpreter.PDFTextState import PDFTextState
from pdfmajor.utils import apply_matrix_norm, apply_matrix_pt, matrix2str, Bbox, Point
from ..PDFGraphicState import PDFColor
from ..PDFTextState import PDFFont
from ..PDFTextState import PDFTextState
## LTCharBlock
##
class LTCharBlock(LTContainer):
def __init__(self,
chars: List[Point],
chars: list, # [str, List[Point]]
textstate: PDFTextState,
ncs: PDFColorSpace,
graphicstate: PDFGraphicState
color: PDFColor
):
((x0, y0), (x1, y1)) = chars[0][1]
c, ((x0, y0), (x1, y1)) = chars[0]
ltchars = [
LTChar(
bbox=Bbox(x0, y0, x1, y1),
char= chars[0][0],
char= c,
textstate=textstate,
ncs=ncs,
graphicstate=graphicstate
color=color
)
]
for char in chars[1:]:
((nx0, ny0), (nx1, ny1)) = char[1]
for c, pts in chars[1:]:
((nx0, ny0), (nx1, ny1)) = pts
x0 = min([ x0, x1, nx0, nx1 ])
x1 = max([ x0, x1, nx0, nx1 ])
y1 = max([ y0, y1, ny0, ny1 ])
y0 = min([ y0, y1, ny0, ny1 ])
ltchars.append(LTChar(
bbox=Bbox(nx0, ny0, nx1, ny1),
char= char[0],
char= c,
textstate=textstate,
ncs=ncs,
graphicstate=graphicstate
color=color
))
LTContainer.__init__(self, Bbox(x0, y0, x1, y1), ltchars)
self.font = textstate.font
self.textstate = textstate
self.ncs = ncs
self.graphicstate = graphicstate
self.color = color
if self.font.is_vertical():
if self.textstate.font.is_vertical():
self.size = self.width
else:
self.size = self.height
@@ -63,6 +57,10 @@ class LTCharBlock(LTContainer):
def get_text(self):
return "".join([ c.get_text() for c in self ])
@property
def font(self):
return self.textstate.font
@property
def fontname(self) -> str:
return self.font.fontname
return self.textstate.font.fontname
@@ -1,9 +1,9 @@
from typing import List, Tuple
from ..utils import Bbox
from ..interpreter.PDFGraphicState import PDFGraphicStateColor
from ..interpreter.types import CurvePath
from pdfmajor.utils import Bbox
from ..PDFGraphicState import PDFColor
from ..Curves import CurvePath
from ._base import LTComponent, Bbox
@@ -13,8 +13,8 @@ class LTCurve(LTComponent):
linewidth: float,
paths: List[CurvePath],
bbox: Bbox,
stroke: PDFGraphicStateColor,
fill: PDFGraphicStateColor,
stroke: PDFColor,
fill: PDFColor,
evenodd: bool,
):
LTComponent.__init__(self, bbox)
@@ -1,5 +1,5 @@
from ..utils import Bbox
from ..interpreter.PDFStream import PDFStream
from pdfmajor.utils import Bbox
from pdfmajor.parser.PDFStream import PDFStream
from ._base import LTComponent
## LTImage
@@ -0,0 +1,22 @@
from typing import List
from pdfmajor.utils import Bbox, Point, matrix2str
from pdfmajor.parser.PDFStream import PDFStream
from ._base import LTContainer
## LTXObject
##
class LTXObject(LTContainer):
def __init__(self, name: str, bbox: Bbox, xobj_stream: PDFStream, resources: dict, t_matrix: List[Point] ):
LTContainer.__init__(self, bbox)
self.name = name
self.stream = xobj_stream
self.t_matrix = t_matrix
self.resources = resources
def __repr__(self):
return ('<%s(%s) %s matrix=%s>' %
(self.__class__.__name__, self.name,
str(self.bbox), matrix2str(self.t_matrix)))
@@ -0,0 +1,133 @@
from typing import List
from pdfmajor.utils import apply_matrix_pt, INF, Bbox, Point, MATRIX_IDENTITY, mult_matrix
from ..Curves import CurvePath
from ..PDFGraphicState import PDFGraphicState, PDFColor
from ..PDFTextState import PDFTextState
from ._base import LTItem, LTComponent, LTContainer
from .LTCharBlock import LTCharBlock, LTChar
from .LTCurves import LTCurve, LTLine, LTHorizontalLine, LTRect, LTVerticalLine
from .LTImage import LTImage
from .LTXObject import LTXObject
from .utils.textdecoder import decode_text_seq
def make_char_block(seq: bytearray, ctm: tuple, textstate: PDFTextState, color: PDFColor):
(char_meta_datas, textstate, color) = decode_text_seq(seq, ctm, textstate, color)
return LTCharBlock(
chars=char_meta_datas,
color=color,
textstate=textstate
)
def make_curve(ctm: tuple, gstate: PDFGraphicState, evenodd: bool, paths: List[CurvePath]):
shape = ''.join(x.method.value for x in paths)
if shape == 'ml':
# horizontal/vertical line
pt0 = paths[0].points[0]
pt1 = paths[1].points[0]
(x0, y0) = apply_matrix_pt(ctm, (pt0.x, pt0.y))
(x1, y1) = apply_matrix_pt(ctm, (pt1.x, pt1.y))
if x0 == x1:
return LTVerticalLine(
linewidth=gstate.linewidth,
paths=paths,
bbox=Bbox( x0, y0, x1, y1 ),
stroke=gstate.scolor,
fill=gstate.ncolor,
evenodd=evenodd,
)
elif y0 == y1:
return LTHorizontalLine(
linewidth=gstate.linewidth,
paths=paths,
bbox=Bbox( x0, y0, x1, y1 ),
stroke=gstate.scolor,
fill=gstate.ncolor,
evenodd=evenodd,
)
else:
return LTLine(
linewidth=gstate.linewidth,
paths=paths,
bbox=Bbox( x0, y0, x1, y1 ),
stroke=gstate.scolor,
fill=gstate.ncolor,
evenodd=evenodd,
)
elif shape == 'mlllh':
# rectangle
pt0 = paths[0].points[0]
pt1 = paths[1].points[0]
pt2 = paths[2].points[0]
pt3 = paths[3].points[0]
(x0, y0) = apply_matrix_pt(ctm, (pt0.x, pt0.y))
(x1, y1) = apply_matrix_pt(ctm, (pt1.x, pt1.y))
(x2, y2) = apply_matrix_pt(ctm, (pt2.x, pt2.y))
(x3, y3) = apply_matrix_pt(ctm, (pt3.x, pt3.y))
if ((x0 == x1 and y1 == y2 and x2 == x3 and y3 == y0) or
(y0 == y1 and x1 == x2 and y2 == y3 and x3 == x0)):
return LTRect(
linewidth=gstate.linewidth,
paths=paths,
bbox=Bbox(
x0=min(x0,x1,x2,x3),
x1=max(x0,x1,x2,x3),
y0=min(y0,y1,y2,y3),
y1=max(y0,y1,y2,y3),
),
evenodd=evenodd,
stroke=gstate.scolor,
fill=gstate.ncolor,
)
else:
(x0, y0, x1, y1) = (INF, INF, -INF, -INF)
for path in paths:
if path.method in [CurvePath.METHOD.MOVE_TO, CurvePath.METHOD.LINE_TO]:
x0 = min(x0, path.points[0].x)
y0 = min(y0, path.points[0].y)
x1 = max(x1, path.points[0].x)
y1 = max(y1, path.points[0].y)
elif path.method in [CurvePath.METHOD.CURVE_BOTH_TO]:
x0 = min(x0, path.points[2].x)
y0 = min(y0, path.points[2].y)
x1 = max(x1, path.points[2].x)
y1 = max(y1, path.points[2].y)
elif path.method in [CurvePath.METHOD.CURVE_NEXT_TO, CurvePath.METHOD.CURVE_FIRST_TO]:
x0 = min(x0, path.points[1].x)
y0 = min(y0, path.points[1].y)
x1 = max(x1, path.points[1].x)
y1 = max(y1, path.points[1].y)
return LTCurve(
linewidth=gstate.linewidth,
paths=paths,
bbox=Bbox(x0,y0,x1,y1),
evenodd=evenodd,
stroke=gstate.scolor,
fill=gstate.ncolor
)
def make_image(obj, ctm):
matrix = mult_matrix(MATRIX_IDENTITY, ctm)
return LTImage(
name=str(id(obj)),
stream=obj,
bbox=Bbox.from_points([
apply_matrix_pt(matrix, (p, q))
for (p, q) in ((0, 0), (1, 0), (0, 1), (1, 1))
])
)
def make_xobject(obj, bbox, ctm, matrix, resources):
(x, y, w, h) = bbox
return LTXObject(
name=str(id(obj)),
bbox=Bbox.from_points([
apply_matrix_pt(matrix, (p, q))
for (p, q) in ((x, y), (x+w, y), (x, y+h), (x+w, y+h))
]),
xobj_stream=obj,
resources=resources,
t_matrix = ctm
)
@@ -1,5 +1,5 @@
from typing import List
from ..utils import Bbox
from pdfmajor.utils import Bbox
## LTItem
##
Whitespace-only changes.
@@ -0,0 +1,90 @@
from pdfmajor.utils import mult_matrix, translate_matrix, Bbox, apply_matrix_pt, isnumber
from ....state import PDFTextState, PDFColor
def decode_text_seq(seq: bytearray, ctm: tuple, textstate: PDFTextState, color: PDFColor):
textstate.matrix = mult_matrix(textstate.matrix, ctm)
textstate.scaling = textstate.scaling * .01
textstate.charspace = textstate.charspace * textstate.scaling
textstate.wordspace = textstate.wordspace * textstate.scaling
if textstate.font.is_multibyte():
textstate.wordspace = 0
dxscale = .001 * textstate.fontsize * textstate.scaling
if textstate.font.is_vertical():
return __render_string_along(1,
seq, textstate, dxscale, color
)
else:
return __render_string_along(0,
seq, textstate, dxscale, color
)
def __render_string_along(idx: int, seq: bytearray,
textstate: PDFTextState, dxscale: float, color: PDFColor):
needcharspace = False
char_meta_datas = []
for obj in seq:
if isnumber(obj):
textstate.linematrix[idx] -= obj*dxscale
needcharspace = True
else:
for cid in textstate.font.decode(obj):
if needcharspace:
textstate.linematrix[idx] += textstate.charspace
text = textstate.font.to_unichr(cid)
assert isinstance(text, str), str(type(text))
matrix = translate_matrix(textstate.matrix, textstate.linematrix)
adv, bbox = __compute_char_bbox(matrix, cid, textstate)
textstate.linematrix[idx] += adv
char_meta_datas.append(
( text, bbox )
)
if cid == 32 and textstate.wordspace:
textstate.linematrix[idx] += textstate.wordspace
needcharspace = True
return (char_meta_datas, textstate, color)
def __compute_char_bbox(matrix, char_id: int, textstate: PDFTextState):
font = textstate.font
fontsize = textstate.fontsize
rise = textstate.rise
scaling = textstate.scaling
textwidth = font.char_width(char_id)
textdisp = font.char_disp(char_id)
adv = textwidth * fontsize * scaling
# compute the boundary rectangle.
if font.is_vertical():
# vertical
width = font.get_width() * fontsize
(vx, vy) = textdisp
if vx is None:
vx = width * 0.5
else:
vx = vx * fontsize * .001
vy = (1000 - vy) * fontsize * .001
tx = -vx
ty = vy + rise
bll = (tx, ty+adv)
bur = (tx+width, ty)
else:
# horizontal
height = font.get_height() * fontsize
descent = font.get_descent() * fontsize
ty = descent + rise
bll = (0, ty)
bur = (adv, ty+height)
(x0, y0) = apply_matrix_pt(matrix, bll)
(x1, y1) = apply_matrix_pt(matrix, bur)
if x1 < x0:
(x0, x1) = (x1, x0)
if y1 < y0:
(y0, y1) = (y1, y0)
return (adv, ((x0, y0), (x1, y1)))
+50
View File
@@ -0,0 +1,50 @@
from pdfmajor.parser.PDFParser import PDFStream
from pdfmajor.parser.PDFStream import list_value, dict_value, resolve1
from pdfmajor.parser.PDFStream.PDFObjRef import PDFObjRef
from pdfmajor.parser.PSStackParser import literal_name
from pdfmajor.parser.constants import LITERAL_PDF, LITERAL_TEXT
from .commands import PDFColorSpace, PREDEFINED_COLORSPACE
from .commands import PDFStateStack, get_font
def get_colorspace(spec):
if isinstance(spec, list):
name = literal_name(spec[0])
else:
name = literal_name(spec)
if name == 'ICCBased' and isinstance(spec, list) and 2 <= len(spec):
return PDFColorSpace(name, PDFStream.validated_stream(spec[1])['N'])
elif name == 'DeviceN' and isinstance(spec, list) and 2 <= len(spec):
return PDFColorSpace(name, len(list_value(spec[1])))
else:
return PREDEFINED_COLORSPACE.get(name)
def get_procset(procs):
for proc in procs:
if proc is LITERAL_PDF:
pass
elif proc is LITERAL_TEXT:
pass
else:
#raise PDFResourceError('ProcSet %r is not supported.' % proc)
pass
return
def init_resources(state: PDFStateStack, font_cache: dict = {}):
for (k, v) in iter(dict_value(state.resources).items()):
# log.debug('Resource: %r: %r', k, v)
if k == 'Font':
for (fontid, spec) in iter(dict_value(v).items()):
objid = None
if isinstance(spec, PDFObjRef):
objid = spec.objid
spec = dict_value(spec)
state.fontmap[fontid] = get_font(objid, spec, font_cache)
elif k == 'ColorSpace':
for (csid, spec) in iter(dict_value(v).items()):
state.colorspace_map[csid] = get_colorspace(resolve1(spec))
elif k == 'ProcSet':
get_procset(list_value(v))
elif k == 'XObject':
for (xobjid, xobjstrm) in iter(dict_value(v).items()):
state.xobjmap[xobjid] = xobjstrm
-18
View File
@@ -1,18 +0,0 @@
from typing import List
from ._base import LTContainer
from ..utils import Bbox, Point, matrix2str
## LTFigure
##
class LTFigure(LTContainer):
def __init__(self, name: str, pts: List[Point], matrix):
LTContainer.__init__(self, Bbox.from_points(pts))
self.name = name
self.matrix = matrix
def __repr__(self):
return ('<%s(%s) %s matrix=%s>' %
(self.__class__.__name__, self.name,
str(self.bbox), matrix2str(self.matrix)))
-20
View File
@@ -1,20 +0,0 @@
from ._base import LTContainer
from ..utils import Bbox
class LTPage(LTContainer):
"""A class reprensting an extracted page from the pdf
Arguments:
pageid {int} -- page number
bbox {Bbox} -- Boundary Box
rotate {float} -- rotation of the page (default: {0})
"""
def __init__(self, pageid: int, bbox: Bbox, rotate: float=0):
LTContainer.__init__(self, bbox)
self.pageid = pageid
self.rotate = rotate
def __repr__(self):
return ('<%s(%r) %s rotate=%r>' %
(self.__class__.__name__, self.pageid,
str(self.bbox), self.rotate))
-189
View File
@@ -1,189 +0,0 @@
import logging
import re
import io
from abc import abstractmethod
from typing import Tuple, List
from ._base import LTItem
from .LTPage import LTPage
from .LTCurves import LTCurve, LTLine, LTRect, LTHorizontalLine, LTVerticalLine
from .LTFigure import LTFigure
from .LTImage import LTImage
from .LTChar import LTChar
from .LTCharBlock import LTCharBlock
from ..utils import Point, Bbox, INF
from ..utils import apply_matrix_pt, mult_matrix
from ..interpreter.PDFFont import PDFUnicodeNotDefined
from ..interpreter.PDFResourceManager import PDFResourceManager
from ..interpreter.PDFGraphicState import PDFGraphicState, PDFGraphicStateColor
from ..interpreter.PDFdevice import PDFTextDevice
from ..interpreter.PDFTextState import PDFTextState
from ..interpreter.PDFColorSpace import PDFColorSpace
from ..interpreter.PDFPage import PDFPage
from ..interpreter.PDFStream import PDFStream
from ..interpreter.types import CurvePath
log = logging.getLogger(__name__)
## PDFLayoutAnalyzer
##
class PDFLayoutAnalyzer(PDFTextDevice):
def __init__(self, rsrcmgr: PDFResourceManager, pageno: int=1):
PDFTextDevice.__init__(self, rsrcmgr)
self.pageno = pageno
self._stack = []
return
def begin_page(self, page: PDFPage, ctm: Tuple[float]):
(x0, y0, x1, y1) = page.mediabox
(x0, y0) = apply_matrix_pt(ctm, (x0, y0))
(x1, y1) = apply_matrix_pt(ctm, (x1, y1))
self.cur_item = LTPage(
self.pageno,
Bbox( 0, 0, abs(x0-x1), abs(y0-y1) )
)
return
def end_page(self, page: PDFPage):
assert not self._stack, str(len(self._stack))
assert isinstance(self.cur_item, LTPage), str(type(self.cur_item))
# self.cur_item.analyze()
self.pageno += 1
self.receive_layout(self.cur_item)
return
def begin_figure(self, name: str, bbox: List[Point], matrix: List[Point]):
self._stack.append(self.cur_item)
matrix = mult_matrix(matrix, self.ctm)
(x, y, w, h) = bbox
self.cur_item = LTFigure(name, [
apply_matrix_pt(matrix, (p, q))
for (p, q) in ((x, y), (x+w, y), (x, y+h), (x+w, y+h))
], matrix)
def end_figure(self, name: str):
fig = self.cur_item
assert isinstance(self.cur_item, LTFigure), str(type(self.cur_item))
self.cur_item = self._stack.pop()
self.cur_item.add(fig)
return
def render_image(self, name: str, stream: PDFStream):
assert isinstance(self.cur_item, LTFigure), str(type(self.cur_item))
item = LTImage(
name,
stream,
Bbox(
self.cur_item.x0, self.cur_item.y0,
self.cur_item.x1, self.cur_item.y1
)
)
self.cur_item.add(item)
return
def paint_path(self, gstate: PDFGraphicState, evenodd: bool, paths: List[CurvePath]):
shape = ''.join(x.method.value for x in paths)
if shape == 'ml':
# horizontal/vertical line
pt0 = paths[0].points[0]
pt1 = paths[1].points[0]
(x0, y0) = apply_matrix_pt(self.ctm, (pt0.x, pt0.y))
(x1, y1) = apply_matrix_pt(self.ctm, (pt1.x, pt1.y))
if x0 == x1:
self.cur_item.add(LTVerticalLine(
linewidth=gstate.linewidth,
paths=paths,
bbox=Bbox( x0, y0, x1, y1 ),
stroke=gstate.scolor,
fill=gstate.ncolor,
evenodd=evenodd,
))
elif y0 == y1:
self.cur_item.add(LTHorizontalLine(
linewidth=gstate.linewidth,
paths=paths,
bbox=Bbox( x0, y0, x1, y1 ),
stroke=gstate.scolor,
fill=gstate.ncolor,
evenodd=evenodd,
))
else:
self.cur_item.add(LTLine(
linewidth=gstate.linewidth,
paths=paths,
bbox=Bbox( x0, y0, x1, y1 ),
stroke=gstate.scolor,
fill=gstate.ncolor,
evenodd=evenodd,
))
elif shape == 'mlllh':
# rectangle
pt0 = paths[0].points[0]
pt1 = paths[1].points[0]
pt2 = paths[2].points[0]
pt3 = paths[3].points[0]
(x0, y0) = apply_matrix_pt(self.ctm, (pt0.x, pt0.y))
(x1, y1) = apply_matrix_pt(self.ctm, (pt1.x, pt1.y))
(x2, y2) = apply_matrix_pt(self.ctm, (pt2.x, pt2.y))
(x3, y3) = apply_matrix_pt(self.ctm, (pt3.x, pt3.y))
if ((x0 == x1 and y1 == y2 and x2 == x3 and y3 == y0) or
(y0 == y1 and x1 == x2 and y2 == y3 and x3 == x0)):
self.cur_item.add(LTRect(
linewidth=gstate.linewidth,
paths=paths,
bbox=Bbox(
x0=min(x0,x1,x2,x3),
x1=max(x0,x1,x2,x3),
y0=min(y0,y1,y2,y3),
y1=max(y0,y1,y2,y3),
),
evenodd=evenodd,
stroke=gstate.scolor,
fill=gstate.ncolor,
))
else:
(x0, y0, x1, y1) = (INF, INF, -INF, -INF)
for path in paths:
if path.method in [CurvePath.METHOD.MOVE_TO, CurvePath.METHOD.LINE_TO]:
x0 = min(x0, path.points[0].x)
y0 = min(y0, path.points[0].y)
x1 = max(x1, path.points[0].x)
y1 = max(y1, path.points[0].y)
elif path.method in [CurvePath.METHOD.CURVE_BOTH_TO]:
x0 = min(x0, path.points[2].x)
y0 = min(y0, path.points[2].y)
x1 = max(x1, path.points[2].x)
y1 = max(y1, path.points[2].y)
elif path.method in [CurvePath.METHOD.CURVE_NEXT_TO, CurvePath.METHOD.CURVE_FIRST_TO]:
x0 = min(x0, path.points[1].x)
y0 = min(y0, path.points[1].y)
x1 = max(x1, path.points[1].x)
y1 = max(y1, path.points[1].y)
self.cur_item.add(LTCurve(
linewidth=gstate.linewidth,
paths=paths,
bbox=Bbox(x0,y0,x1,y1),
evenodd=evenodd,
stroke=gstate.scolor,
fill=gstate.ncolor
))
def render_char_box(self, chars: list, textstate: PDFTextState, graphicstate: PDFGraphicState, ncs: PDFColorSpace):
self.cur_item.add(LTCharBlock(
chars=chars,
textstate=textstate,
ncs=ncs,
graphicstate=graphicstate
))
@abstractmethod
def receive_layout(self, ltpage: LTPage):
raise NotImplementedError
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -91,12 +91,12 @@ class PDFPage(object):
tree_type = tree.get('type')
if tree_type is LITERAL_PAGES and 'Kids' in tree:
# log.info('Pages: Kids=%r', tree['Kids'])
log.debug('Pages: Kids=%r', tree['Kids'])
for c in list_value(tree['Kids']):
for x in search(c, tree):
yield x
elif tree_type is LITERAL_PAGE:
log.info('Page: %r', tree)
log.debug('Page: %r', tree)
yield (objid, tree)
pages = False
File renamed without changes.
Whitespace-only changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -95,6 +95,14 @@ class PSStackParser(PSBaseParser):
def do_keyword(self, pos, token):
return
def __iter__(self):
while True:
try:
(_, obj) = self.nextobject()
yield obj
except PSEOF:
break
def nextobject(self):
"""Yields a list of objects.
View File
Whitespace-only changes.
File renamed without changes.
File renamed without changes.
+27
View File
@@ -0,0 +1,27 @@
from enum import Enum
from typing import List
class CurvePoint:
def __init__(self, x: float, y: float):
self.x = x
self.y = y
def __repr__(self) -> str:
return f'<CurvePoint ({self.x}, {self.y})/>'
class CurveMethod(Enum):
MOVE_TO='m'
LINE_TO='l'
CURVE_BOTH_TO='c'
CURVE_NEXT_TO='v'
CURVE_FIRST_TO='y'
CLOSE_PATH='h'
class CurvePath:
METHOD = CurveMethod
def __init__(self, method: CurveMethod, *points: List[CurvePoint]):
self.method = method
self.points = points
def __repr__(self) -> str:
return f'<CurvePath {self.method} path=[{",".join(map(repr, self.points))}]/>'
-45
View File
@@ -1,45 +0,0 @@
#!/usr/bin/env python
import re
import os
import logging
from typing import List
from pdfmajor.converters import convert_file
from xml.etree import ElementTree
from tqdm import tqdm
def rel_path(*arg: List[str]) -> str:
return os.path.join(os.path.dirname(__file__), *arg)
EXT = 'html'
INPUT_FOLDER = rel_path("./pdf")
OUTPUT_FOLDER = rel_path(f"./output/{EXT}")
if not os.path.exists(OUTPUT_FOLDER):
os.makedirs(OUTPUT_FOLDER)
files = os.listdir(INPUT_FOLDER)
TOTAL = len(files)
waiter = tqdm(enumerate(files), total=TOTAL)
for idx, file_name in waiter:
file_name_no_ext = re.sub(r'\.pdf$',"", file_name)
input_file_path = os.path.join(INPUT_FOLDER, file_name)
output_file_path = os.path.join(OUTPUT_FOLDER, file_name_no_ext + "." + EXT)
if len(file_name) > len(file_name_no_ext):
waiter.set_description(f"Processing {file_name}")
with open(input_file_path, 'rb') as input_file:
with open(output_file_path, 'wb') as output_file:
convert_file(
input_file=input_file,
output_file=output_file,
out_type=EXT,
debug_level=logging.NOTSET,
)
if EXT == 'xml':
waiter.set_description(f" > testing format {file_name}")
ElementTree.parse(output_file_path)
waiter.refresh()
waiter.set_description("Done")
waiter.close()
-23
View File
@@ -1,23 +0,0 @@
#!/usr/bin/env python
import re
import os
import logging
from typing import List
from pdfmajor.converters import convert_file
from xml.etree import ElementTree
from tqdm import tqdm
from pdfmajor.extractor import extract_items_from_pdf
from pdfmajor.layouts import LTPage
def rel_path(*arg: List[str]) -> str:
return os.path.join(os.path.dirname(__file__), *arg)
print(extract_items_from_pdf(rel_path('pdf/chars.pdf')))
# for page in extract_items_from_pdf(rel_path('pdf/chars.pdf')):
# print('page-start', page)
# for item in page:
# print(' ', item)
# print('page-end', page)
Binary file not shown.
+1 -1
View File
@@ -10,7 +10,7 @@ import sys
with open('README.md') as readme_file:
SETUP = {
"name": "pdfmajor",
"version": "1.2.0",
"version": "1.3.0",
"packages": find_packages(include=['pdfmajor*']),
"install_requires": [
'pycryptodome',
View File
File renamed without changes.
File renamed without changes.
File renamed without changes.
Loaded 100 of 113 files, more files were not shown because too many files have changed in this diff. Show more