CalvinBackup/ScreenCoder_UI2Code
1
0
Fork 0
mirror of https://github.com/leigest519/ScreenCoder.git synced 2026-02-12 17:52:47 +00:00
Code Issues Packages Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Yilei JIANG
2025-07-28 18:43:47 +08:00
parent 0a6e4ab682
commit 06408ffa6a
702 changed files with 153932 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
.gitignore vendored Normal file
View File

@@ -0,0 +1,11 @@
# Node / JavaScript
node_modules/
dist/
.DS_Store
# Environments
.env
.venv
# API keys
doubao_api.txt

89
README.md Normal file
View File

@@ -0,0 +1,89 @@
# Screencoder
### About
This is the Screencoder Project. Screencoder generates the HTML code for a website screenshot using a modular multi-agent framework.
### Project Structure
- `main.py`: The main script to generate final HTML code for a single screenshot.
- `UIED/`: Contains the UIED (UI Element Detection) engine for analyzing screenshots and detecting components.
- `run_single.py`: Python script to run UI component detection on a single image.
- `html_generator.py`: Takes the detected component data and generates a complete HTML layout with generated code for each module.
- `image_replacer.py`: A script to replace placeholder divs in the final HTML with actual cropped images.
- `mapping.py`: Maps the detected UIED components to logical page regions.
- `requirements.txt`: Lists all the necessary Python dependencies for the project.
- `doubao_api.txt`: API key file for the Doubao model (should be kept private and is included in `.gitignore`).
### Setup and Installation
1. **Clone the repository:**
```bash
git clone https://github.com/JimmyZhengyz/screencoder.git
cd screencoder
```
2. **Create a virtual environment:**
```bash
python3 -m venv .venv
source .venv/bin/activate
```
3. **Install dependencies:**
```bash
pip install -r requirements.txt
```
4. **Set up API Key:**
- Create a file named `doubao_api.txt` in the root directory.
- Paste your Doubao API key into this file.
### Usage
The typical workflow is a multi-step process as follows:
1. **Initial Generation with Placeholders:**
Run the Python script to generate the initial HTML code for a given screenshot.
- Block Detection:
```bash
python block_parsor.py
```
- Generation with Placeholders (Gray Images Blocks):
```bash
python html_generator.py
```
2. **Final HTML Code:**
Run the python script to generate final HTML code with copped images from the original screenshot.
- Placeholder Detection:
```bash
python image_box_detection.py
```
- UI Element Detection:
```bash
python UIED/run_single.py
```
- Mapping Alignment Between Placeholders and UI Elements:
```bash
python mapping.py
```
- Placeholder Replacement:
```bash
python image_replacer.py
```
3. **Simple Run:**
Run the python script to generate the final HTML code:
```bash
python main.py
```
### Demo
To access demo, you can follow these steps:
```bash
cd demo
pnpm install
pnpm run dev
```
Then, you can see the demo running.

BIN
UIED.zip Normal file
View File

Binary file not shown.

14
UIED/.idea/UIED.iml generated Normal file
View File

@@ -0,0 +1,14 @@
<?xml version="1.0" encoding="UTF-8"?>
<module type="PYTHON_MODULE" version="4">
<component name="NewModuleRootManager">
<content url="file://$MODULE_DIR$">
<sourceFolder url="file://$MODULE_DIR$" isTestSource="false" />
<sourceFolder url="file://$MODULE_DIR$/resnet" isTestSource="false" />
</content>
<orderEntry type="inheritedJdk" />
<orderEntry type="sourceFolder" forTests="false" />
</component>
<component name="TestRunnerService">
<option name="PROJECT_TEST_RUNNER" value="py.test" />
</component>
</module>

29
UIED/.idea/inspectionProfiles/Project_Default.xml generated Normal file
View File

@@ -0,0 +1,29 @@
<component name="InspectionProjectProfileManager">
<profile version="1.0">
<option name="myName" value="Project Default" />
<inspection_tool class="DuplicatedCode" enabled="true" level="WEAK WARNING" enabled_by_default="true">
<Languages>
<language minSize="54" name="Python" />
</Languages>
</inspection_tool>
<inspection_tool class="PyInterpreterInspection" enabled="false" level="WARNING" enabled_by_default="false" />
<inspection_tool class="PyPackageRequirementsInspection" enabled="true" level="WARNING" enabled_by_default="true">
<option name="ignoredPackages">
<value>
<list size="3">
<item index="0" class="java.lang.String" itemvalue="Tensorflow" />
<item index="1" class="java.lang.String" itemvalue="Sklearn" />
<item index="2" class="java.lang.String" itemvalue="Opencv" />
</list>
</value>
</option>
</inspection_tool>
<inspection_tool class="PyPep8Inspection" enabled="true" level="WEAK WARNING" enabled_by_default="true">
<option name="ignoredErrors">
<list>
<option value="E501" />
</list>
</option>
</inspection_tool>
</profile>
</component>

4
UIED/.idea/misc.xml generated Normal file
View File

@@ -0,0 +1,4 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="ProjectRootManager" version="2" project-jdk-name="Python 3.6" project-jdk-type="Python SDK" />
</project>

8
UIED/.idea/modules.xml generated Normal file
View File

@@ -0,0 +1,8 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="ProjectModuleManager">
<modules>
<module fileurl="file://$PROJECT_DIR$/.idea/UIED.iml" filepath="$PROJECT_DIR$/.idea/UIED.iml" />
</modules>
</component>
</project>

6
UIED/.idea/vcs.xml generated Normal file
View File

@@ -0,0 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="VcsDirectoryMappings">
<mapping directory="$PROJECT_DIR$" vcs="Git" />
</component>
</project>

336
UIED/.idea/workspace.xml generated Normal file
View File

@@ -0,0 +1,336 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="AutoImportSettings">
<option name="autoReloadType" value="SELECTIVE" />
</component>
<component name="ChangeListManager">
<list default="true" id="b4069649-920d-465f-ac6b-bac85007c2bb" name="Default" comment="">
<change beforePath="$PROJECT_DIR$/.idea/workspace.xml" beforeDir="false" afterPath="$PROJECT_DIR$/.idea/workspace.xml" afterDir="false" />
</list>
<option name="SHOW_DIALOG" value="false" />
<option name="HIGHLIGHT_CONFLICTS" value="true" />
<option name="HIGHLIGHT_NON_ACTIVE_CHANGELIST" value="false" />
<option name="LAST_RESOLUTION" value="IGNORE" />
</component>
<component name="FileTemplateManagerImpl">
<option name="RECENT_TEMPLATES">
<list>
<option value="Python Script" />
</list>
</option>
</component>
<component name="Git.Settings">
<option name="RECENT_GIT_ROOT_PATH" value="$PROJECT_DIR$" />
</component>
<component name="JupyterTrust" id="2ce0fe3c-0081-4dca-a6a6-b1219d650764" />
<component name="ProjectId" id="1dwCihBTog6GQX95sgxr7TpM6ZO" />
<component name="ProjectLevelVcsManager">
<ConfirmationsSetting value="2" id="Add" />
</component>
<component name="ProjectViewState">
<option name="hideEmptyMiddlePackages" value="true" />
<option name="showLibraryContents" value="true" />
<option name="showMembers" value="true" />
</component>
<component name="PropertiesComponent">
<property name="RunOnceActivity.OpenProjectViewOnStart" value="true" />
<property name="RunOnceActivity.ShowReadmeOnStart" value="true" />
<property name="WebServerToolWindowFactoryState" value="false" />
<property name="last_opened_file_path" value="$PROJECT_DIR$" />
<property name="node.js.detected.package.eslint" value="true" />
<property name="node.js.detected.package.tslint" value="true" />
<property name="node.js.path.for.package.eslint" value="project" />
<property name="node.js.path.for.package.tslint" value="project" />
<property name="node.js.selected.package.eslint" value="(autodetect)" />
<property name="node.js.selected.package.tslint" value="(autodetect)" />
<property name="restartRequiresConfirmation" value="false" />
<property name="settings.editor.selected.configurable" value="com.jetbrains.python.configuration.PyActiveSdkModuleConfigurable" />
<property name="two.files.diff.last.used.file" value="$PROJECT_DIR$/../UI2CODE/Element-Detection/merge.py" />
</component>
<component name="RecentsManager">
<key name="CopyFile.RECENT_KEYS">
<recent name="D:\git_file\github\doing\UIED\data\demo" />
</key>
<key name="MoveFile.RECENT_KEYS">
<recent name="D:\git_file\github\doing\UIED\detect_compo" />
<recent name="D:\git_file\github\doing\UIED\detect_compo\deprecated" />
<recent name="D:\git_file\github\doing\UIED\utils" />
<recent name="D:\git_file\github\doing\UIED" />
<recent name="D:\git_file\github\doing\UIED\result_processing" />
</key>
</component>
<component name="RunManager" selected="Python.run_single">
<configuration name="merge" type="PythonConfigurationType" factoryName="Python" temporary="true" nameIsGenerated="true">
<module name="UIED" />
<option name="INTERPRETER_OPTIONS" value="" />
<option name="PARENT_ENVS" value="true" />
<envs>
<env name="PYTHONUNBUFFERED" value="1" />
</envs>
<option name="SDK_HOME" value="" />
<option name="WORKING_DIRECTORY" value="$PROJECT_DIR$/detect_merge" />
<option name="IS_MODULE_SDK" value="true" />
<option name="ADD_CONTENT_ROOTS" value="true" />
<option name="ADD_SOURCE_ROOTS" value="true" />
<EXTENSION ID="PythonCoverageRunConfigurationExtension" runner="coverage.py" />
<option name="SCRIPT_NAME" value="$PROJECT_DIR$/detect_merge/merge.py" />
<option name="PARAMETERS" value="" />
<option name="SHOW_COMMAND_LINE" value="false" />
<option name="EMULATE_TERMINAL" value="false" />
<option name="MODULE_MODE" value="false" />
<option name="REDIRECT_INPUT" value="false" />
<option name="INPUT_FILE" value="" />
<method v="2" />
</configuration>
<configuration name="merge2" type="PythonConfigurationType" factoryName="Python" temporary="true" nameIsGenerated="true">
<module name="UIED" />
<option name="INTERPRETER_OPTIONS" value="" />
<option name="PARENT_ENVS" value="true" />
<envs>
<env name="PYTHONUNBUFFERED" value="1" />
</envs>
<option name="SDK_HOME" value="" />
<option name="WORKING_DIRECTORY" value="$PROJECT_DIR$" />
<option name="IS_MODULE_SDK" value="true" />
<option name="ADD_CONTENT_ROOTS" value="true" />
<option name="ADD_SOURCE_ROOTS" value="true" />
<EXTENSION ID="PythonCoverageRunConfigurationExtension" runner="coverage.py" />
<option name="SCRIPT_NAME" value="$PROJECT_DIR$/merge2.py" />
<option name="PARAMETERS" value="" />
<option name="SHOW_COMMAND_LINE" value="false" />
<option name="EMULATE_TERMINAL" value="false" />
<option name="MODULE_MODE" value="false" />
<option name="REDIRECT_INPUT" value="false" />
<option name="INPUT_FILE" value="" />
<method v="2" />
</configuration>
<configuration name="run_single" type="PythonConfigurationType" factoryName="Python" temporary="true" nameIsGenerated="true">
<module name="UIED" />
<option name="INTERPRETER_OPTIONS" value="" />
<option name="PARENT_ENVS" value="true" />
<envs>
<env name="PYTHONUNBUFFERED" value="1" />
</envs>
<option name="SDK_HOME" value="" />
<option name="WORKING_DIRECTORY" value="$PROJECT_DIR$" />
<option name="IS_MODULE_SDK" value="true" />
<option name="ADD_CONTENT_ROOTS" value="true" />
<option name="ADD_SOURCE_ROOTS" value="true" />
<EXTENSION ID="PythonCoverageRunConfigurationExtension" runner="coverage.py" />
<option name="SCRIPT_NAME" value="$PROJECT_DIR$/run_single.py" />
<option name="PARAMETERS" value="" />
<option name="SHOW_COMMAND_LINE" value="false" />
<option name="EMULATE_TERMINAL" value="false" />
<option name="MODULE_MODE" value="false" />
<option name="REDIRECT_INPUT" value="false" />
<option name="INPUT_FILE" value="" />
<method v="2" />
</configuration>
<configuration name="run_testing(Used for Adjusting)" type="PythonConfigurationType" factoryName="Python" temporary="true" nameIsGenerated="true">
<module name="UIED" />
<option name="INTERPRETER_OPTIONS" value="" />
<option name="PARENT_ENVS" value="true" />
<envs>
<env name="PYTHONUNBUFFERED" value="1" />
</envs>
<option name="SDK_HOME" value="" />
<option name="WORKING_DIRECTORY" value="$PROJECT_DIR$" />
<option name="IS_MODULE_SDK" value="true" />
<option name="ADD_CONTENT_ROOTS" value="true" />
<option name="ADD_SOURCE_ROOTS" value="true" />
<EXTENSION ID="PythonCoverageRunConfigurationExtension" runner="coverage.py" />
<option name="SCRIPT_NAME" value="$PROJECT_DIR$/run_testing(Used for Adjusting).py" />
<option name="PARAMETERS" value="" />
<option name="SHOW_COMMAND_LINE" value="false" />
<option name="EMULATE_TERMINAL" value="false" />
<option name="MODULE_MODE" value="false" />
<option name="REDIRECT_INPUT" value="false" />
<option name="INPUT_FILE" value="" />
<method v="2" />
</configuration>
<configuration name="text_detection" type="PythonConfigurationType" factoryName="Python" temporary="true" nameIsGenerated="true">
<module name="UIED" />
<option name="INTERPRETER_OPTIONS" value="" />
<option name="PARENT_ENVS" value="true" />
<envs>
<env name="PYTHONUNBUFFERED" value="1" />
</envs>
<option name="SDK_HOME" value="" />
<option name="WORKING_DIRECTORY" value="$PROJECT_DIR$/detect_text" />
<option name="IS_MODULE_SDK" value="true" />
<option name="ADD_CONTENT_ROOTS" value="true" />
<option name="ADD_SOURCE_ROOTS" value="true" />
<EXTENSION ID="PythonCoverageRunConfigurationExtension" runner="coverage.py" />
<option name="SCRIPT_NAME" value="$PROJECT_DIR$/detect_text/text_detection.py" />
<option name="PARAMETERS" value="" />
<option name="SHOW_COMMAND_LINE" value="false" />
<option name="EMULATE_TERMINAL" value="false" />
<option name="MODULE_MODE" value="false" />
<option name="REDIRECT_INPUT" value="false" />
<option name="INPUT_FILE" value="" />
<method v="2" />
</configuration>
<list>
<item itemvalue="Python.merge2" />
<item itemvalue="Python.merge" />
<item itemvalue="Python.run_single" />
<item itemvalue="Python.run_testing(Used for Adjusting)" />
<item itemvalue="Python.text_detection" />
</list>
<recent_temporary>
<list>
<item itemvalue="Python.run_single" />
<item itemvalue="Python.run_testing(Used for Adjusting)" />
<item itemvalue="Python.merge" />
<item itemvalue="Python.merge2" />
<item itemvalue="Python.text_detection" />
</list>
</recent_temporary>
</component>
<component name="SpellCheckerSettings" RuntimeDictionaries="0" Folders="0" CustomDictionaries="0" DefaultDictionary="application-level" UseSingleDictionary="true" transferred="true" />
<component name="SvnConfiguration">
<configuration />
</component>
<component name="TaskManager">
<task active="true" id="Default" summary="Default task">
<changelist id="b4069649-920d-465f-ac6b-bac85007c2bb" name="Default" comment="" />
<created>1581826543105</created>
<option name="number" value="Default" />
<option name="presentableId" value="Default" />
<updated>1581826543105</updated>
<workItem from="1593326898629" duration="4378000" />
<workItem from="1594035929667" duration="39000" />
<workItem from="1594085137817" duration="1275000" />
<workItem from="1594100418101" duration="3473000" />
<workItem from="1594163935256" duration="1077000" />
<workItem from="1594182213679" duration="2275000" />
<workItem from="1595462115111" duration="3597000" />
<workItem from="1595466120556" duration="3158000" />
<workItem from="1595488240502" duration="74000" />
<workItem from="1595723287741" duration="773000" />
<workItem from="1596417993986" duration="2972000" />
<workItem from="1596442796733" duration="9601000" />
<workItem from="1596604406529" duration="10000" />
<workItem from="1596685531295" duration="1526000" />
<workItem from="1596792120495" duration="27485000" />
<workItem from="1596926803756" duration="894000" />
<workItem from="1596955557805" duration="717000" />
<workItem from="1597015667916" duration="1488000" />
<workItem from="1598487885922" duration="962000" />
<workItem from="1601609000493" duration="599000" />
<workItem from="1601849117964" duration="1226000" />
<workItem from="1601850762269" duration="6477000" />
<workItem from="1601933366434" duration="6847000" />
<workItem from="1602130037944" duration="4303000" />
<workItem from="1602199380249" duration="5493000" />
<workItem from="1603669721746" duration="6042000" />
<workItem from="1604011435077" duration="517000" />
<workItem from="1604016832655" duration="5114000" />
<workItem from="1604037397074" duration="12109000" />
<workItem from="1604564719252" duration="1133000" />
<workItem from="1604619358289" duration="17569000" />
<workItem from="1604874207809" duration="171000" />
<workItem from="1605071062104" duration="3079000" />
<workItem from="1605086142565" duration="6397000" />
<workItem from="1625010508533" duration="4541000" />
<workItem from="1625099073176" duration="49720000" />
<workItem from="1625443415902" duration="26350000" />
<workItem from="1625529598285" duration="26479000" />
<workItem from="1625613709029" duration="14000" />
<workItem from="1625730508694" duration="928000" />
<workItem from="1625809233064" duration="837000" />
<workItem from="1626009011038" duration="18000" />
<workItem from="1626307428798" duration="1983000" />
<workItem from="1628054466383" duration="1894000" />
<workItem from="1628122812217" duration="7049000" />
<workItem from="1630237947629" duration="453000" />
<workItem from="1630268189943" duration="20000" />
<workItem from="1630297231550" duration="5710000" />
<workItem from="1630312264694" duration="7894000" />
<workItem from="1631149307515" duration="752000" />
<workItem from="1631576239206" duration="805000" />
<workItem from="1631584649434" duration="2272000" />
<workItem from="1648024033251" duration="1883000" />
</task>
<servers />
</component>
<component name="TestHistory">
<history-entry file="py_test_in_test_lucky_py - 2020.03.04 at 05h 51m 04s.xml">
<configuration name="py.test in test_lucky.py" configurationId="tests" />
</history-entry>
</component>
<component name="TypeScriptGeneratedFilesManager">
<option name="version" value="3" />
</component>
<component name="XDebuggerManager">
<breakpoint-manager>
<breakpoints>
<line-breakpoint enabled="true" suspend="THREAD" type="python-line">
<url />
<line>147</line>
<option name="timeStamp" value="3" />
</line-breakpoint>
<line-breakpoint enabled="true" suspend="THREAD" type="python-line">
<url />
<line>134</line>
<option name="timeStamp" value="4" />
</line-breakpoint>
<line-breakpoint enabled="true" suspend="THREAD" type="python-line">
<url />
<line>135</line>
<option name="timeStamp" value="5" />
</line-breakpoint>
<line-breakpoint enabled="true" suspend="THREAD" type="python-line">
<url />
<line>136</line>
<option name="timeStamp" value="6" />
</line-breakpoint>
<line-breakpoint enabled="true" suspend="THREAD" type="python-line">
<url>file://$PROJECT_DIR$/detect_text_east/lib_east/eval.py</url>
<line>263</line>
<option name="timeStamp" value="67" />
</line-breakpoint>
<line-breakpoint enabled="true" suspend="THREAD" type="python-line">
<url>file://$PROJECT_DIR$/result_processing/eval_classes.py</url>
<line>92</line>
<option name="timeStamp" value="92" />
</line-breakpoint>
<line-breakpoint enabled="true" suspend="THREAD" type="python-line">
<url>file://$PROJECT_DIR$/detect_text_east/lib_east/eval.py</url>
<line>108</line>
<option name="timeStamp" value="93" />
</line-breakpoint>
<line-breakpoint enabled="true" suspend="THREAD" type="python-line">
<url>file://$PROJECT_DIR$/cnn/CNN.py</url>
<line>62</line>
<option name="timeStamp" value="94" />
</line-breakpoint>
<line-breakpoint enabled="true" suspend="THREAD" type="python-line">
<url>file://$PROJECT_DIR$/run_single.py</url>
<line>27</line>
<option name="timeStamp" value="101" />
</line-breakpoint>
</breakpoints>
<default-breakpoints>
<breakpoint type="python-exception">
<properties notifyOnTerminate="true" exception="BaseException">
<option name="notifyOnTerminate" value="true" />
</properties>
</breakpoint>
</default-breakpoints>
</breakpoint-manager>
</component>
<component name="com.intellij.coverage.CoverageDataManagerImpl">
<SUITE FILE_PATH="coverage/UIED$view_gt.coverage" NAME="view_gt Coverage Results" MODIFIED="1596418105849" SOURCE_PROVIDER="com.intellij.coverage.DefaultCoverageFileProvider" RUNNER="coverage.py" COVERAGE_BY_TEST_ENABLED="true" COVERAGE_TRACING_ENABLED="false" WORKING_DIRECTORY="$PROJECT_DIR$/result_processing" />
<SUITE FILE_PATH="coverage/UIED$run_single.coverage" NAME="run_single Coverage Results" MODIFIED="1648024874423" SOURCE_PROVIDER="com.intellij.coverage.DefaultCoverageFileProvider" RUNNER="coverage.py" COVERAGE_BY_TEST_ENABLED="true" COVERAGE_TRACING_ENABLED="false" WORKING_DIRECTORY="$PROJECT_DIR$" />
<SUITE FILE_PATH="coverage/UIED$main_single.coverage" NAME="main_single Coverage Results" MODIFIED="1594102734340" SOURCE_PROVIDER="com.intellij.coverage.DefaultCoverageFileProvider" RUNNER="coverage.py" COVERAGE_BY_TEST_ENABLED="true" COVERAGE_TRACING_ENABLED="false" WORKING_DIRECTORY="$PROJECT_DIR$" />
<SUITE FILE_PATH="coverage/UIED$run_batch.coverage" NAME="run_batch Coverage Results" MODIFIED="1596448499254" SOURCE_PROVIDER="com.intellij.coverage.DefaultCoverageFileProvider" RUNNER="coverage.py" COVERAGE_BY_TEST_ENABLED="true" COVERAGE_TRACING_ENABLED="false" WORKING_DIRECTORY="$PROJECT_DIR$" />
<SUITE FILE_PATH="coverage/UIED_block$main_single.coverage" NAME="main_single Coverage Results" MODIFIED="1594035942350" SOURCE_PROVIDER="com.intellij.coverage.DefaultCoverageFileProvider" RUNNER="coverage.py" COVERAGE_BY_TEST_ENABLED="true" COVERAGE_TRACING_ENABLED="false" WORKING_DIRECTORY="$PROJECT_DIR$" />
<SUITE FILE_PATH="coverage/UIED$run_testing_Used_for_Adjusting_.coverage" NAME="run_testing(Used for Adjusting) Coverage Results" MODIFIED="1631149318891" SOURCE_PROVIDER="com.intellij.coverage.DefaultCoverageFileProvider" RUNNER="coverage.py" COVERAGE_BY_TEST_ENABLED="true" COVERAGE_TRACING_ENABLED="false" WORKING_DIRECTORY="$PROJECT_DIR$" />
<SUITE FILE_PATH="coverage/UIED$merge2.coverage" NAME="merge2 Coverage Results" MODIFIED="1625271385465" SOURCE_PROVIDER="com.intellij.coverage.DefaultCoverageFileProvider" RUNNER="coverage.py" COVERAGE_BY_TEST_ENABLED="true" COVERAGE_TRACING_ENABLED="false" WORKING_DIRECTORY="$PROJECT_DIR$" />
<SUITE FILE_PATH="coverage/UIED$run_testing_Use_Me_for_Adjusting_.coverage" NAME="run_testing(Use Me for Adjusting) Coverage Results" MODIFIED="1605091364353" SOURCE_PROVIDER="com.intellij.coverage.DefaultCoverageFileProvider" RUNNER="coverage.py" COVERAGE_BY_TEST_ENABLED="true" COVERAGE_TRACING_ENABLED="false" WORKING_DIRECTORY="$PROJECT_DIR$" />
<SUITE FILE_PATH="coverage/UIED$text_detection.coverage" NAME="text_detection Coverage Results" MODIFIED="1625107691453" SOURCE_PROVIDER="com.intellij.coverage.DefaultCoverageFileProvider" RUNNER="coverage.py" COVERAGE_BY_TEST_ENABLED="true" COVERAGE_TRACING_ENABLED="false" WORKING_DIRECTORY="$PROJECT_DIR$/detect_text" />
<SUITE FILE_PATH="coverage/UIED$merge.coverage" NAME="merge Coverage Results" MODIFIED="1625284362497" SOURCE_PROVIDER="com.intellij.coverage.DefaultCoverageFileProvider" RUNNER="coverage.py" COVERAGE_BY_TEST_ENABLED="true" COVERAGE_TRACING_ENABLED="false" WORKING_DIRECTORY="$PROJECT_DIR$/detect_merge" />
<SUITE FILE_PATH="coverage/UIED$experiment.coverage" NAME="experiment Coverage Results" MODIFIED="1605087951044" SOURCE_PROVIDER="com.intellij.coverage.DefaultCoverageFileProvider" RUNNER="coverage.py" COVERAGE_BY_TEST_ENABLED="true" COVERAGE_TRACING_ENABLED="false" WORKING_DIRECTORY="$PROJECT_DIR$/result_processing" />
</component>
</project>

201
UIED/LICENSE Normal file
View File

@@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [2021] [UIED mulong.xie@anu.edu.au]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

80
UIED/README.md Normal file
View File

@@ -0,0 +1,80 @@
# UIED - UI element detection, detecting UI elements from UI screenshots or drawnings
This project is still ongoing and this repo may be updated irregularly, I developed a web app for the UIED in http://uied.online
## Related Publications:
[1. UIED: a hybrid tool for GUI element detection](https://dl.acm.org/doi/10.1145/3368089.3417940)
[2. Object Detection for Graphical User Interface: Old Fashioned or Deep Learning or a Combination?](https://arxiv.org/abs/2008.05132)
>The repo has been **upgraded with Google OCR** for GUI text detection, to use the original version in our paper (using [EAST](https://github.com/argman/EAST) as text detector), check the relase [v2.3](https://github.com/MulongXie/UIED/releases/tag/v2.3) and download the pre-trained model in [this link](https://drive.google.com/drive/folders/1MK0Om7Lx0wRXGDfNcyj21B0FL1T461v5?usp=sharing).
## What is it?
UI Element Detection (UIED) is an old-fashioned computer vision (CV) based element detection approach for graphic user interface.
The input of UIED could be various UI image, such as mobile app or web page screenshot, UI design drawn by Photoshop or Sketch, and even some hand-drawn UI design. Then the approach detects and classifies text and graphic UI elements, and exports the detection result as JSON file for future application.
UIED comprises two parts to detect UI text and graphic elements, such as button, image and input bar.
* For text, it leverages [Google OCR](https://cloud.google.com/vision/docs/ocr) to perfrom detection.
* For graphical elements, it uses old-fashioned CV approaches to locate the elements and a CNN classifier to achieve classification.
> UIED is highly customizable, you can replace both parts by your choice (e.g. other text detection approaches). Unlike black-box end-to-end deep learning approach, you can revise the algorithms in the non-text detection and merging (partially or entirely) easily to fit your task.
![UIED Approach](https://github.com/MulongXie/UIED/blob/master/data/demo/approach.png)
## How to use?
### Dependency
* **Python 3.5**
* **Opencv 3.4.2**
* **Pandas**
<!-- * **Tensorflow 1.10.0**
* **Keras 2.2.4**
* **Sklearn 0.22.2** -->
### Installation
<!-- Install the mentioned dependencies, and download two pre-trained models from [this link](https://drive.google.com/drive/folders/1MK0Om7Lx0wRXGDfNcyj21B0FL1T461v5?usp=sharing) for EAST text detection and GUI element classification. -->
<!-- Change ``CNN_PATH`` and ``EAST_PATH`` in *config/CONFIG.py* to your locations. -->
The new version of UIED equipped with Google OCR is easy to deploy and no pre-trained model is needed. Simply donwload the repo along with the dependencies.
> Please replace the Google OCR key at `detect_text/ocr.py line 28` with your own (apply in [Google website](https://cloud.google.com/vision)).
### Usage
To test your own image(s):
* To test single image, change *input_path_img* in ``run_single.py`` to your input image and the results will be output to *output_root*.
* To test mutiple images, change *input_img_root* in ``run_batch.py`` to your input directory and the results will be output to *output_root*.
* To adjust the parameters lively, using ``run_testing.py``
> Note: The best set of parameters vary for different types of GUI image (Mobile App, Web, PC). I highly recommend to first play with the ``run_testing.py`` to pick a good set of parameters for your data.
## Folder structure
``cnn/``
* Used to train classifier for graphic UI elements
* Set path of the CNN classification model
``config/``
* Set data paths
* Set parameters for graphic elements detection
``data/``
* Input UI images and output detection results
``detect_compo/``
* Non-text GUI component detection
``detect_text/``
* GUI text detection using Google OCR
``detect_merge/``
* Merge the detection results of non-text and text GUI elements
The major detection algorithms are in ``detect_compo/``, ``detect_text/`` and ``detect_merge/``
## Demo
GUI element detection result for web screenshot
![UI Components detection result](https://github.com/MulongXie/UIED/blob/master/data/demo/demo.png)

125
UIED/cnn/CNN.py Normal file
View File

@@ -0,0 +1,125 @@
import keras
from keras.applications.resnet50 import ResNet50
from keras.models import Model,load_model
from keras.layers import Dense, Activation, Flatten, Dropout
from sklearn.metrics import confusion_matrix
import numpy as np
import cv2
from config.CONFIG import Config
cfg = Config()
class CNN:
def __init__(self, classifier_type, is_load=True):
'''
:param classifier_type: 'Text' or 'Noise' or 'Elements'
'''
self.data = None
self.model = None
self.classifier_type = classifier_type
self.image_shape = (32,32,3)
self.class_number = None
self.class_map = None
self.model_path = None
self.classifier_type = classifier_type
if is_load:
self.load(classifier_type)
def build_model(self, epoch_num, is_compile=True):
base_model = ResNet50(include_top=False, weights='imagenet', input_shape=self.image_shape)
for layer in base_model.layers:
layer.trainable = False
self.model = Flatten()(base_model.output)
self.model = Dense(128, activation='relu')(self.model)
self.model = Dropout(0.5)(self.model)
self.model = Dense(15, activation='softmax')(self.model)
self.model = Model(inputs=base_model.input, outputs=self.model)
if is_compile:
self.model.compile(loss='categorical_crossentropy', optimizer='adadelta', metrics=['accuracy'])
self.model.fit(self.data.X_train, self.data.Y_train, batch_size=64, epochs=epoch_num, verbose=1,
validation_data=(self.data.X_test, self.data.Y_test))
def train(self, data, epoch_num=30):
self.data = data
self.build_model(epoch_num)
self.model.save(self.model_path)
print("Trained model is saved to", self.model_path)
def load(self, classifier_type):
if classifier_type == 'Text':
self.model_path = 'E:/Mulong/Model/rico_compos/cnn-textview-2.h5'
self.class_map = ['Text', 'Non-Text']
elif classifier_type == 'Noise':
self.model_path = 'E:/Mulong/Model/rico_compos/cnn-noise-1.h5'
self.class_map = ['Noise', 'Non-Noise']
elif classifier_type == 'Elements':
# self.model_path = 'E:/Mulong/Model/rico_compos/resnet-ele14-19.h5'
# self.model_path = 'E:/Mulong/Model/rico_compos/resnet-ele14-28.h5'
# self.model_path = 'E:/Mulong/Model/rico_compos/resnet-ele14-45.h5'
self.model_path = 'UIED/cnn/model/cnn-rico-1.h5' # Use local model
self.class_map = cfg.element_class
self.image_shape = (64, 64, 3)
elif classifier_type == 'Image':
# Redirect 'Image' classification to use the general 'Elements' model
# as the specific model is not available in the project.
# IMPORTANT: This requires the actual model file to be present for real classification.
print("Warning: 'Image' specific model not found. Redirecting to general 'Elements' classifier.")
self.model_path = 'UIED/cnn/model/cnn-rico-1.h5' # Use local model
self.class_map = ['Image', 'Non-Image'] # Keep the class map for binary classification logic
self.class_number = len(self.class_map)
try:
self.model = load_model(self.model_path)
print('Model Loaded From', self.model_path)
except Exception as e:
print(f"Error loading model: {e}")
print("A dummy model file was created, but it's not a valid Keras model.")
print("Please replace it with the actual model file for classification to work.")
self.model = None
def preprocess_img(self, image):
image = cv2.resize(image, self.image_shape[:2])
x = (image / 255).astype('float32')
x = np.array([x])
return x
def predict(self, imgs, compos, load=False, show=False):
"""
:type img_path: list of img path
"""
if load:
self.load(self.classifier_type)
if self.model is None:
print("*** No model loaded ***")
return
for i in range(len(imgs)):
X = self.preprocess_img(imgs[i])
Y = self.class_map[np.argmax(self.model.predict(X))]
compos[i].category = Y
if show:
print(Y)
cv2.imshow('element', imgs[i])
cv2.waitKey()
def evaluate(self, data, load=True):
if load:
self.load(self.classifier_type)
X_test = data.X_test
Y_test = [np.argmax(y) for y in data.Y_test]
Y_pre = [np.argmax(y_pre) for y_pre in self.model.predict(X_test, verbose=1)]
matrix = confusion_matrix(Y_test, Y_pre)
print(matrix)
TP, FP, FN = 0, 0, 0
for i in range(len(matrix)):
TP += matrix[i][i]
FP += sum(matrix[i][:]) - matrix[i][i]
FN += sum(matrix[:][i]) - matrix[i][i]
precision = TP/(TP+FP)
recall = TP / (TP+FN)
print("Precision:%.3f, Recall:%.3f" % (precision, recall))

21
UIED/cnn/Config.py Normal file
View File

@@ -0,0 +1,21 @@
class Config:
def __init__(self):
# cnn 4 classes
# self.MODEL_PATH = 'E:/Mulong/Model/ui_compos/cnn6_icon.h5' # cnn 4 classes
# self.class_map = ['Image', 'Icon', 'Button', 'Input']
# resnet 14 classes
# self.DATA_PATH = "E:/Mulong/Datasets/rico/elements-14-2"
# self.MODEL_PATH = 'E:/Mulong/Model/rico_compos/resnet-ele14.h5'
# self.class_map = ['Button', 'CheckBox', 'Chronometer', 'EditText', 'ImageButton', 'ImageView',
# 'ProgressBar', 'RadioButton', 'RatingBar', 'SeekBar', 'Spinner', 'Switch',
# 'ToggleButton', 'VideoView', 'TextView'] # ele-14
self.DATA_PATH = "E:\Mulong\Datasets\dataset_webpage\Components3"
self.MODEL_PATH = 'E:/Mulong/Model/rico_compos/cnn2-textview.h5'
self.class_map = ['Text', 'Non-Text']
self.image_shape = (32, 32, 3)
self.class_number = len(self.class_map)

69
UIED/cnn/Data.py Normal file
View File

@@ -0,0 +1,69 @@
import cv2
import numpy as np
from os.path import join as pjoin
import glob
from tqdm import tqdm
from Config import Config
cfg = Config()
class Data:
def __init__(self):
self.data_num = 0
self.images = []
self.labels = []
self.X_train, self.Y_train = None, None
self.X_test, self.Y_test = None, None
self.image_shape = cfg.image_shape
self.class_number = cfg.class_number
self.class_map = cfg.class_map
self.DATA_PATH = cfg.DATA_PATH
def load_data(self, resize=True, shape=None, max_number=1000000):
# if customize shape
if shape is not None:
self.image_shape = shape
else:
shape = self.image_shape
# load data
for p in glob.glob(pjoin(self.DATA_PATH, '*')):
print("*** Loading components of %s: %d ***" %(p.split('\\')[-1], int(len(glob.glob(pjoin(p, '*.png'))))))
label = self.class_map.index(p.split('\\')[-1]) # map to index of classes
for i, image_path in enumerate(tqdm(glob.glob(pjoin(p, '*.png'))[:max_number])):
image = cv2.imread(image_path)
if resize:
image = cv2.resize(image, shape[:2])
self.images.append(image)
self.labels.append(label)
assert len(self.images) == len(self.labels)
self.data_num = len(self.images)
print('%d Data Loaded' % self.data_num)
def generate_training_data(self, train_data_ratio=0.8):
# transfer int into c dimensions one-hot array
def expand(label, class_number):
# return y : (num_class, num_samples)
y = np.eye(class_number)[label]
y = np.squeeze(y)
return y
# reshuffle
np.random.seed(0)
self.images = np.random.permutation(self.images)
np.random.seed(0)
self.labels = np.random.permutation(self.labels)
Y = expand(self.labels, self.class_number)
# separate dataset
cut = int(train_data_ratio * self.data_num)
self.X_train = (self.images[:cut] / 255).astype('float32')
self.X_test = (self.images[cut:] / 255).astype('float32')
self.Y_train = Y[:cut]
self.Y_test = Y[cut:]
print('X_train:%d, Y_train:%d' % (len(self.X_train), len(self.Y_train)))
print('X_test:%d, Y_test:%d' % (len(self.X_test), len(self.Y_test)))

BIN
UIED/cnn/__pycache__/CNN.cpython-312.pyc Normal file
View File

Binary file not shown.

BIN
UIED/cnn/__pycache__/CNN.cpython-35.pyc Normal file
View File

Binary file not shown.

0
UIED/cnn/model/cnn-rico-1.h5 Normal file
View File

45
UIED/config/CONFIG.py Normal file
View File

@@ -0,0 +1,45 @@
from os.path import join as pjoin
import os
class Config:
def __init__(self):
# setting CNN (graphic elements) model
self.image_shape = (64, 64, 3)
# self.MODEL_PATH = 'E:\\Mulong\\Model\\UI2CODE\\cnn6_icon.h5'
# self.class_map = ['button', 'input', 'icon', 'img', 'text']
self.CNN_PATH = 'E:/Mulong/Model/rico_compos/cnn-rico-1.h5'
self.element_class = ['Button', 'CheckBox', 'Chronometer', 'EditText', 'ImageButton', 'ImageView',
'ProgressBar', 'RadioButton', 'RatingBar', 'SeekBar', 'Spinner', 'Switch',
'ToggleButton', 'VideoView', 'TextView']
self.class_number = len(self.element_class)
# setting EAST (ocr) model
self.EAST_PATH = 'E:/Mulong/Model/East/east_icdar2015_resnet_v1_50_rbox'
self.COLOR = {'Button': (0, 255, 0), 'CheckBox': (0, 0, 255), 'Chronometer': (255, 166, 166),
'EditText': (255, 166, 0),
'ImageButton': (77, 77, 255), 'ImageView': (255, 0, 166), 'ProgressBar': (166, 0, 255),
'RadioButton': (166, 166, 166),
'RatingBar': (0, 166, 255), 'SeekBar': (0, 166, 10), 'Spinner': (50, 21, 255),
'Switch': (80, 166, 66), 'ToggleButton': (0, 66, 80), 'VideoView': (88, 66, 0),
'TextView': (169, 255, 0), 'NonText': (0,0,255),
'Compo':(0, 0, 255), 'Text':(169, 255, 0), 'Block':(80, 166, 66)}
def build_output_folders(self):
# setting data flow paths
self.ROOT_INPUT = "E:\\Mulong\\Datasets\\rico\\combined"
self.ROOT_OUTPUT = "E:\\Mulong\\Result\\rico\\rico_uied\\rico_new_uied_v3"
self.ROOT_IMG_ORG = pjoin(self.ROOT_INPUT, "org")
self.ROOT_IP = pjoin(self.ROOT_OUTPUT, "ip")
self.ROOT_OCR = pjoin(self.ROOT_OUTPUT, "ocr")
self.ROOT_MERGE = pjoin(self.ROOT_OUTPUT, "merge")
self.ROOT_IMG_COMPONENT = pjoin(self.ROOT_OUTPUT, "components")
if not os.path.exists(self.ROOT_IP):
os.mkdir(self.ROOT_IP)
if not os.path.exists(self.ROOT_OCR):
os.mkdir(self.ROOT_OCR)
if not os.path.exists(self.ROOT_MERGE):
os.mkdir(self.ROOT_MERGE)

49
UIED/config/CONFIG_UIED.py Normal file
View File

@@ -0,0 +1,49 @@
class Config:
def __init__(self):
# Adjustable
# self.THRESHOLD_PRE_GRADIENT = 4 # dribbble:4 rico:4 web:1
# self.THRESHOLD_OBJ_MIN_AREA = 55 # bottom line 55 of small circle
# self.THRESHOLD_BLOCK_GRADIENT = 5
# *** Frozen ***
self.THRESHOLD_REC_MIN_EVENNESS = 0.7
self.THRESHOLD_REC_MAX_DENT_RATIO = 0.25
self.THRESHOLD_LINE_THICKNESS = 8
self.THRESHOLD_LINE_MIN_LENGTH = 0.95
self.THRESHOLD_COMPO_MAX_SCALE = (0.25, 0.98) # (120/800, 422.5/450) maximum height and width ratio for a atomic compo (button)
self.THRESHOLD_TEXT_MAX_WORD_GAP = 10
self.THRESHOLD_TEXT_MAX_HEIGHT = 0.04 # 40/800 maximum height of text
self.THRESHOLD_TOP_BOTTOM_BAR = (0.045, 0.94) # (36/800, 752/800) height ratio of top and bottom bar
self.THRESHOLD_BLOCK_MIN_HEIGHT = 0.03 # 24/800
# deprecated
# self.THRESHOLD_OBJ_MIN_PERIMETER = 0
# self.THRESHOLD_BLOCK_MAX_BORDER_THICKNESS = 8
# self.THRESHOLD_BLOCK_MAX_CROSS_POINT = 0.1
# self.THRESHOLD_UICOMPO_MIN_W_H_RATIO = 0.4
# self.THRESHOLD_TEXT_MAX_WIDTH = 150
# self.THRESHOLD_LINE_MIN_LENGTH_H = 50
# self.THRESHOLD_LINE_MIN_LENGTH_V = 50
# self.OCR_PADDING = 5
# self.OCR_MIN_WORD_AREA = 0.45
# self.THRESHOLD_MIN_IOU = 0.1 # dribbble:0.003 rico:0.1 web:0.1
# self.THRESHOLD_BLOCK_MIN_EDGE_LENGTH = 210 # dribbble:68 rico:210 web:70
# self.THRESHOLD_UICOMPO_MAX_W_H_RATIO = 10 # dribbble:10 rico:10 web:22
self.CLASS_MAP = {'0':'Button', '1':'CheckBox', '2':'Chronometer', '3':'EditText', '4':'ImageButton', '5':'ImageView',
'6':'ProgressBar', '7':'RadioButton', '8':'RatingBar', '9':'SeekBar', '10':'Spinner', '11':'Switch',
'12':'ToggleButton', '13':'VideoView', '14':'TextView'}
self.COLOR = {'Button': (0, 255, 0), 'CheckBox': (0, 0, 255), 'Chronometer': (255, 166, 166),
'EditText': (255, 166, 0),
'ImageButton': (77, 77, 255), 'ImageView': (255, 0, 166), 'ProgressBar': (166, 0, 255),
'RadioButton': (166, 166, 166),
'RatingBar': (0, 166, 255), 'SeekBar': (0, 166, 10), 'Spinner': (50, 21, 255),
'Switch': (80, 166, 66), 'ToggleButton': (0, 66, 80), 'VideoView': (88, 66, 0),
'TextView': (169, 255, 0),
'Text':(169, 255, 0), 'Non-Text':(255, 0, 166),
'Noise':(6,6,255), 'Non-Noise': (6,255,6),
'Image':(255,6,6), 'Non-Image':(6,6,255)}

BIN
UIED/config/__pycache__/CONFIG.cpython-312.pyc Normal file
View File

Binary file not shown.

BIN
UIED/config/__pycache__/CONFIG.cpython-35.pyc Normal file
View File

Binary file not shown.

BIN
UIED/config/__pycache__/CONFIG.cpython-37.pyc Normal file
View File

Binary file not shown.

BIN
UIED/config/__pycache__/CONFIG_UIED.cpython-312.pyc Normal file
View File

Binary file not shown.

BIN
UIED/config/__pycache__/CONFIG_UIED.cpython-35.pyc Normal file
View File

Binary file not shown.

BIN
UIED/config/__pycache__/CONFIG_UIED.cpython-37.pyc Normal file
View File

Binary file not shown.

BIN
UIED/data/demo/approach.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.9 MiB

BIN
UIED/data/demo/demo.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.6 MiB

BIN
UIED/data/input/0.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1020 KiB

BIN
UIED/data/input/1.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 60 KiB

BIN
UIED/data/input/10.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 123 KiB

BIN
UIED/data/input/100.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 108 KiB

BIN
UIED/data/input/11.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 167 KiB

BIN
UIED/data/input/11300.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 491 KiB

BIN
UIED/data/input/1220.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 264 KiB

BIN
UIED/data/input/1565.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 520 KiB

BIN
UIED/data/input/1627.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 464 KiB

BIN
UIED/data/input/18116.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 603 KiB

BIN
UIED/data/input/2.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 97 KiB

BIN
UIED/data/input/214.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 116 KiB

BIN
UIED/data/input/24.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 802 KiB

BIN
UIED/data/input/245.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 276 KiB

BIN
UIED/data/input/3.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 158 KiB

BIN
UIED/data/input/30800.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 351 KiB

BIN
UIED/data/input/4.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 421 KiB

BIN
UIED/data/input/413.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 63 KiB

BIN
UIED/data/input/472.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 159 KiB

BIN
UIED/data/input/472a.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 409 KiB

BIN
UIED/data/input/493.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 141 KiB

BIN
UIED/data/input/497.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 155 KiB

BIN
UIED/data/input/5.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 34 KiB

BIN
UIED/data/input/505.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 223 KiB

BIN
UIED/data/input/6.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 324 KiB

BIN
UIED/data/input/66529.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 396 KiB

BIN
UIED/data/input/7.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 217 KiB

BIN
UIED/data/input/8.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 155 KiB

BIN
UIED/data/input/9.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 464 KiB

BIN
UIED/data/input/9.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.0 MiB

BIN
UIED/data/input/a.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 57 KiB

BIN
UIED/data/input/b.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 57 KiB

BIN
UIED/data/input/d.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 69 KiB

BIN
UIED/data/input/e.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 128 KiB

BIN
UIED/data/input/f.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 52 KiB

BIN
UIED/data/input/g.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 105 KiB

BIN
UIED/data/input/x.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 74 KiB

BIN
UIED/data/input/y.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 40 KiB

BIN
UIED/data/input/z.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 56 KiB

BIN
UIED/data/output/ip/1565.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 146 KiB

639
UIED/data/output/ip/1565.json Normal file
View File

@@ -0,0 +1,639 @@
{
"compos": [
{
"column_min": 20,
"row_max": 44,
"id": 1,
"column_max": 49,
"width": 29,
"height": 21,
"class": "Compo",
"row_min": 23
},
{
"column_min": 95,
"row_max": 45,
"id": 2,
"column_max": 126,
"width": 31,
"height": 22,
"class": "Compo",
"row_min": 23
},
{
"column_min": 398,
"row_max": 45,
"id": 3,
"column_max": 422,
"width": 24,
"height": 23,
"class": "Compo",
"row_min": 22
},
{
"column_min": 452,
"row_max": 45,
"id": 4,
"column_max": 487,
"width": 35,
"height": 22,
"class": "Compo",
"row_min": 23
},
{
"column_min": 346,
"row_max": 38,
"id": 5,
"column_max": 360,
"width": 14,
"height": 8,
"class": "Compo",
"row_min": 30
},
{
"column_min": 0,
"row_max": 282,
"id": 6,
"column_max": 498,
"width": 498,
"height": 215,
"class": "Compo",
"row_min": 67
},
{
"column_min": 135,
"row_max": 327,
"id": 7,
"column_max": 228,
"width": 93,
"height": 22,
"class": "Compo",
"row_min": 305
},
{
"column_min": 233,
"row_max": 327,
"id": 8,
"column_max": 293,
"width": 60,
"height": 22,
"class": "Compo",
"row_min": 305
},
{
"column_min": 17,
"row_max": 407,
"id": 9,
"column_max": 117,
"width": 100,
"height": 101,
"class": "Compo",
"row_min": 306
},
{
"column_min": 298,
"row_max": 333,
"id": 10,
"column_max": 365,
"width": 67,
"height": 25,
"class": "Compo",
"row_min": 308
},
{
"column_min": 368,
"row_max": 327,
"id": 11,
"column_max": 416,
"width": 48,
"height": 17,
"class": "Compo",
"row_min": 310
},
{
"column_min": 133,
"row_max": 357,
"id": 12,
"column_max": 265,
"width": 132,
"height": 23,
"class": "Compo",
"row_min": 334
},
{
"column_min": 269,
"row_max": 354,
"id": 13,
"column_max": 376,
"width": 107,
"height": 20,
"class": "Compo",
"row_min": 334
},
{
"column_min": 134,
"row_max": 385,
"id": 14,
"column_max": 173,
"width": 39,
"height": 20,
"class": "Compo",
"row_min": 365
},
{
"column_min": 177,
"row_max": 385,
"id": 15,
"column_max": 269,
"width": 92,
"height": 21,
"class": "Compo",
"row_min": 364
},
{
"column_min": 295,
"row_max": 380,
"id": 16,
"column_max": 332,
"width": 37,
"height": 16,
"class": "Compo",
"row_min": 364
},
{
"column_min": 334,
"row_max": 381,
"id": 17,
"column_max": 390,
"width": 56,
"height": 17,
"class": "Compo",
"row_min": 364
},
{
"column_min": 402,
"row_max": 380,
"id": 18,
"column_max": 436,
"width": 34,
"height": 16,
"class": "Compo",
"row_min": 364
},
{
"column_min": 273,
"row_max": 385,
"id": 19,
"column_max": 276,
"width": 3,
"height": 21,
"class": "Compo",
"row_min": 364
},
{
"column_min": 281,
"row_max": 380,
"id": 20,
"column_max": 291,
"width": 10,
"height": 14,
"class": "Compo",
"row_min": 366
},
{
"column_min": 394,
"row_max": 382,
"id": 21,
"column_max": 397,
"width": 3,
"height": 16,
"class": "Compo",
"row_min": 366
},
{
"column_min": 16,
"row_max": 547,
"id": 22,
"column_max": 117,
"width": 101,
"height": 103,
"class": "Compo",
"row_min": 444
},
{
"column_min": 132,
"row_max": 490,
"id": 23,
"column_max": 238,
"width": 106,
"height": 44,
"class": "Compo",
"row_min": 446
},
{
"column_min": 240,
"row_max": 469,
"id": 24,
"column_max": 346,
"width": 106,
"height": 23,
"class": "Compo",
"row_min": 446
},
{
"column_min": 351,
"row_max": 468,
"id": 25,
"column_max": 367,
"width": 16,
"height": 22,
"class": "Compo",
"row_min": 446
},
{
"column_min": 372,
"row_max": 465,
"id": 26,
"column_max": 385,
"width": 13,
"height": 15,
"class": "Compo",
"row_min": 450
},
{
"column_min": 354,
"row_max": 490,
"id": 27,
"column_max": 371,
"width": 17,
"height": 21,
"class": "Compo",
"row_min": 469
},
{
"column_min": 243,
"row_max": 494,
"id": 28,
"column_max": 348,
"width": 105,
"height": 23,
"class": "Compo",
"row_min": 471
},
{
"column_min": 376,
"row_max": 490,
"id": 29,
"column_max": 419,
"width": 43,
"height": 15,
"class": "Compo",
"row_min": 475
},
{
"column_min": 133,
"row_max": 520,
"id": 30,
"column_max": 203,
"width": 70,
"height": 23,
"class": "Compo",
"row_min": 497
},
{
"column_min": 134,
"row_max": 546,
"id": 31,
"column_max": 160,
"width": 26,
"height": 19,
"class": "Compo",
"row_min": 527
},
{
"column_min": 164,
"row_max": 547,
"id": 32,
"column_max": 232,
"width": 68,
"height": 20,
"class": "Compo",
"row_min": 527
},
{
"column_min": 236,
"row_max": 545,
"id": 33,
"column_max": 240,
"width": 4,
"height": 17,
"class": "Compo",
"row_min": 528
},
{
"column_min": 244,
"row_max": 545,
"id": 34,
"column_max": 254,
"width": 10,
"height": 19,
"class": "Compo",
"row_min": 526
},
{
"column_min": 258,
"row_max": 544,
"id": 35,
"column_max": 353,
"width": 95,
"height": 18,
"class": "Compo",
"row_min": 526
},
{
"column_min": 357,
"row_max": 546,
"id": 36,
"column_max": 361,
"width": 4,
"height": 19,
"class": "Compo",
"row_min": 527
},
{
"column_min": 364,
"row_max": 543,
"id": 37,
"column_max": 384,
"width": 20,
"height": 16,
"class": "Compo",
"row_min": 527
},
{
"column_min": 295,
"row_max": 602,
"id": 38,
"column_max": 324,
"width": 29,
"height": 22,
"class": "Compo",
"row_min": 580
},
{
"column_min": 16,
"row_max": 681,
"id": 39,
"column_max": 117,
"width": 101,
"height": 100,
"class": "Compo",
"row_min": 581
},
{
"column_min": 132,
"row_max": 604,
"id": 40,
"column_max": 171,
"width": 39,
"height": 21,
"class": "Compo",
"row_min": 583
},
{
"column_min": 177,
"row_max": 605,
"id": 41,
"column_max": 219,
"width": 42,
"height": 24,
"class": "Compo",
"row_min": 581
},
{
"column_min": 221,
"row_max": 608,
"id": 42,
"column_max": 291,
"width": 70,
"height": 21,
"class": "Compo",
"row_min": 587
},
{
"column_min": 328,
"row_max": 607,
"id": 43,
"column_max": 372,
"width": 44,
"height": 20,
"class": "Compo",
"row_min": 587
},
{
"column_min": 132,
"row_max": 632,
"id": 44,
"column_max": 228,
"width": 96,
"height": 24,
"class": "Compo",
"row_min": 608
},
{
"column_min": 230,
"row_max": 629,
"id": 45,
"column_max": 279,
"width": 49,
"height": 20,
"class": "Compo",
"row_min": 609
},
{
"column_min": 280,
"row_max": 628,
"id": 46,
"column_max": 310,
"width": 30,
"height": 20,
"class": "Compo",
"row_min": 608
},
{
"column_min": 133,
"row_max": 659,
"id": 47,
"column_max": 189,
"width": 56,
"height": 21,
"class": "Compo",
"row_min": 638
},
{
"column_min": 220,
"row_max": 659,
"id": 48,
"column_max": 232,
"width": 12,
"height": 23,
"class": "Compo",
"row_min": 636
},
{
"column_min": 244,
"row_max": 657,
"id": 49,
"column_max": 253,
"width": 9,
"height": 18,
"class": "Compo",
"row_min": 639
},
{
"column_min": 257,
"row_max": 656,
"id": 50,
"column_max": 352,
"width": 95,
"height": 18,
"class": "Compo",
"row_min": 638
},
{
"column_min": 193,
"row_max": 658,
"id": 51,
"column_max": 218,
"width": 25,
"height": 18,
"class": "Compo",
"row_min": 640
},
{
"column_min": 235,
"row_max": 657,
"id": 52,
"column_max": 239,
"width": 4,
"height": 16,
"class": "Compo",
"row_min": 641
},
{
"column_min": 363,
"row_max": 655,
"id": 53,
"column_max": 383,
"width": 20,
"height": 14,
"class": "Compo",
"row_min": 641
},
{
"column_min": 17,
"row_max": 799,
"id": 54,
"column_max": 116,
"width": 99,
"height": 80,
"class": "Compo",
"row_min": 719
},
{
"column_min": 132,
"row_max": 768,
"id": 55,
"column_max": 307,
"width": 175,
"height": 49,
"class": "Compo",
"row_min": 719
},
{
"column_min": 307,
"row_max": 770,
"id": 56,
"column_max": 423,
"width": 116,
"height": 50,
"class": "Compo",
"row_min": 720
},
{
"column_min": 135,
"row_max": 793,
"id": 57,
"column_max": 200,
"width": 65,
"height": 15,
"class": "Compo",
"row_min": 778
},
{
"column_min": 205,
"row_max": 793,
"id": 58,
"column_max": 228,
"width": 23,
"height": 15,
"class": "Compo",
"row_min": 778
},
{
"column_min": 232,
"row_max": 795,
"id": 59,
"column_max": 236,
"width": 4,
"height": 16,
"class": "Compo",
"row_min": 779
},
{
"column_min": 241,
"row_max": 793,
"id": 60,
"column_max": 250,
"width": 9,
"height": 15,
"class": "Compo",
"row_min": 778
},
{
"column_min": 254,
"row_max": 793,
"id": 61,
"column_max": 349,
"width": 95,
"height": 15,
"class": "Compo",
"row_min": 778
},
{
"column_min": 353,
"row_max": 795,
"id": 62,
"column_max": 357,
"width": 4,
"height": 16,
"class": "Compo",
"row_min": 779
},
{
"column_min": 361,
"row_max": 793,
"id": 63,
"column_max": 380,
"width": 19,
"height": 15,
"class": "Compo",
"row_min": 778
}
],
"img_shape": [
800,
499,
3
]
}

BIN
UIED/data/output/ip/497.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 118 KiB

1059
UIED/data/output/ip/497.json Normal file
View File

File diff suppressed because it is too large Load Diff

BIN
UIED/data/output/merge/1565.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 144 KiB

436
UIED/data/output/merge/1565.json Normal file
View File

@@ -0,0 +1,436 @@
{
"compos": [
{
"height": 21,
"width": 29,
"position": {
"column_min": 20,
"row_max": 44,
"column_max": 49,
"row_min": 23
},
"class": "Compo",
"id": 0
},
{
"height": 23,
"width": 24,
"position": {
"column_min": 398,
"row_max": 45,
"column_max": 422,
"row_min": 22
},
"class": "Compo",
"id": 1
},
{
"height": 22,
"width": 35,
"position": {
"column_min": 452,
"row_max": 45,
"column_max": 487,
"row_min": 23
},
"class": "Compo",
"id": 2
},
{
"height": 8,
"width": 14,
"position": {
"column_min": 346,
"row_max": 38,
"column_max": 360,
"row_min": 30
},
"class": "Compo",
"id": 3
},
{
"height": 215,
"width": 498,
"position": {
"column_min": 0,
"row_max": 282,
"column_max": 498,
"row_min": 67
},
"class": "Compo",
"id": 4
},
{
"height": 101,
"width": 100,
"position": {
"column_min": 17,
"row_max": 407,
"column_max": 117,
"row_min": 306
},
"class": "Compo",
"id": 5
},
{
"height": 21,
"width": 3,
"position": {
"column_min": 273,
"row_max": 385,
"column_max": 276,
"row_min": 364
},
"class": "Compo",
"id": 6
},
{
"height": 14,
"width": 10,
"position": {
"column_min": 281,
"row_max": 380,
"column_max": 291,
"row_min": 366
},
"class": "Compo",
"id": 7
},
{
"height": 103,
"width": 101,
"position": {
"column_min": 16,
"row_max": 547,
"column_max": 117,
"row_min": 444
},
"class": "Compo",
"id": 8
},
{
"height": 17,
"width": 4,
"position": {
"column_min": 236,
"row_max": 545,
"column_max": 240,
"row_min": 528
},
"class": "Compo",
"id": 9
},
{
"height": 19,
"width": 10,
"position": {
"column_min": 244,
"row_max": 545,
"column_max": 254,
"row_min": 526
},
"class": "Compo",
"id": 10
},
{
"height": 100,
"width": 101,
"position": {
"column_min": 16,
"row_max": 681,
"column_max": 117,
"row_min": 581
},
"class": "Compo",
"id": 11
},
{
"height": 18,
"width": 9,
"position": {
"column_min": 244,
"row_max": 657,
"column_max": 253,
"row_min": 639
},
"class": "Compo",
"id": 12
},
{
"height": 16,
"width": 4,
"position": {
"column_min": 235,
"row_max": 657,
"column_max": 239,
"row_min": 641
},
"class": "Compo",
"id": 13
},
{
"height": 80,
"width": 99,
"position": {
"column_min": 17,
"row_max": 799,
"column_max": 116,
"row_min": 719
},
"class": "Compo",
"id": 14
},
{
"position": {
"column_min": 96,
"row_max": 45,
"column_max": 124,
"row_min": 24
},
"class": "Text",
"text_content": "ALL",
"id": 15,
"width": 28,
"height": 21
},
{
"position": {
"column_min": 137,
"row_max": 329,
"column_max": 415,
"row_min": 304
},
"class": "Text",
"text_content": "Microsoft thinks people want",
"id": 16,
"width": 278,
"height": 25
},
{
"position": {
"column_min": 135,
"row_max": 357,
"column_max": 373,
"row_min": 328
},
"class": "Text",
"text_content": "ultra portable headaches",
"id": 17,
"width": 238,
"height": 29
},
{
"position": {
"column_min": 137,
"row_max": 382,
"column_max": 270,
"row_min": 363
},
"class": "Text",
"text_content": "Jerry Hildenbrand",
"id": 18,
"width": 133,
"height": 19
},
{
"position": {
"column_min": 297,
"row_max": 382,
"column_max": 435,
"row_min": 363
},
"class": "Text",
"text_content": "COMMENTS 57m",
"id": 19,
"width": 138,
"height": 19
},
{
"position": {
"column_min": 137,
"row_max": 470,
"column_max": 383,
"row_min": 443
},
"class": "Text",
"text_content": "My Disney Experience is a",
"id": 20,
"width": 246,
"height": 27
},
{
"position": {
"column_min": 136,
"row_max": 493,
"column_max": 415,
"row_min": 469
},
"class": "Text",
"text_content": "whole new experience in new",
"id": 21,
"width": 279,
"height": 24
},
{
"position": {
"column_min": 136,
"row_max": 518,
"column_max": 201,
"row_min": 494
},
"class": "Text",
"text_content": "update",
"id": 22,
"width": 65,
"height": 24
},
{
"position": {
"column_min": 136,
"row_max": 547,
"column_max": 231,
"row_min": 525
},
"class": "Text",
"text_content": "Ara Wagoner",
"id": 23,
"width": 95,
"height": 22
},
{
"position": {
"column_min": 259,
"row_max": 547,
"column_max": 383,
"row_min": 525
},
"class": "Text",
"text_content": "COMMENTS 3h",
"id": 24,
"width": 124,
"height": 22
},
{
"position": {
"column_min": 136,
"row_max": 605,
"column_max": 218,
"row_min": 578
},
"class": "Text",
"text_content": "The best",
"id": 25,
"width": 82,
"height": 27
},
{
"position": {
"column_min": 226,
"row_max": 630,
"column_max": 322,
"row_min": 579
},
"class": "Text",
"text_content": "games VR for Gear",
"id": 26,
"width": 96,
"height": 51
},
{
"position": {
"column_min": 329,
"row_max": 607,
"column_max": 370,
"row_min": 580
},
"class": "Text",
"text_content": "your",
"id": 27,
"width": 41,
"height": 27
},
{
"position": {
"column_min": 135,
"row_max": 632,
"column_max": 224,
"row_min": 605
},
"class": "Text",
"text_content": "Samsung",
"id": 28,
"width": 89,
"height": 27
},
{
"position": {
"column_min": 136,
"row_max": 660,
"column_max": 230,
"row_min": 637
},
"class": "Text",
"text_content": "Russell Holly",
"id": 29,
"width": 94,
"height": 23
},
{
"position": {
"column_min": 259,
"row_max": 660,
"column_max": 382,
"row_min": 637
},
"class": "Text",
"text_content": "COMMENTS 4h",
"id": 30,
"width": 123,
"height": 23
},
{
"position": {
"column_min": 137,
"row_max": 742,
"column_max": 421,
"row_min": 717
},
"class": "Text",
"text_content": "Here's how to get a little more",
"id": 31,
"width": 284,
"height": 25
},
{
"position": {
"column_min": 136,
"row_max": 768,
"column_max": 395,
"row_min": 743
},
"class": "Text",
"text_content": "Android Central in your life !",
"id": 32,
"width": 259,
"height": 25
},
{
"position": {
"column_min": 136,
"row_max": 796,
"column_max": 379,
"row_min": 776
},
"class": "Text",
"text_content": "Florence lon5 COMMENTS 5h",
"id": 33,
"width": 243,
"height": 20
}
],
"img_shape": [
800,
499,
3
]
}

BIN
UIED/data/output/merge/497.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 116 KiB

598
UIED/data/output/merge/497.json Normal file
View File

@@ -0,0 +1,598 @@
{
"compos": [
{
"height": 39,
"class": "Compo",
"width": 369,
"id": 0,
"position": {
"row_max": 73,
"column_max": 379,
"column_min": 10,
"row_min": 34
}
},
{
"height": 62,
"class": "Compo",
"width": 67,
"id": 1,
"position": {
"row_max": 153,
"column_max": 88,
"column_min": 21,
"row_min": 91
}
},
{
"height": 12,
"class": "Compo",
"width": 9,
"id": 2,
"position": {
"row_max": 140,
"column_max": 165,
"column_min": 156,
"row_min": 128
}
},
{
"height": 66,
"class": "Compo",
"width": 67,
"id": 3,
"position": {
"row_max": 230,
"column_max": 88,
"column_min": 21,
"row_min": 164
}
},
{
"height": 12,
"class": "Compo",
"width": 9,
"id": 4,
"position": {
"row_max": 217,
"column_max": 165,
"column_min": 156,
"row_min": 205
}
},
{
"height": 66,
"class": "Compo",
"width": 67,
"id": 5,
"position": {
"row_max": 307,
"column_max": 88,
"column_min": 21,
"row_min": 241
}
},
{
"height": 15,
"class": "Compo",
"width": 16,
"id": 6,
"position": {
"row_max": 297,
"column_max": 109,
"column_min": 93,
"row_min": 282
}
},
{
"height": 14,
"class": "Compo",
"width": 10,
"id": 7,
"position": {
"row_max": 296,
"column_max": 166,
"column_min": 156,
"row_min": 282
}
},
{
"height": 69,
"class": "Compo",
"width": 69,
"id": 8,
"position": {
"row_max": 387,
"column_max": 89,
"column_min": 20,
"row_min": 318
}
},
{
"height": 12,
"class": "Compo",
"width": 15,
"id": 9,
"position": {
"row_max": 372,
"column_max": 108,
"column_min": 93,
"row_min": 360
}
},
{
"height": 13,
"class": "Compo",
"width": 12,
"id": 10,
"position": {
"row_max": 372,
"column_max": 162,
"column_min": 150,
"row_min": 359
}
},
{
"height": 68,
"class": "Compo",
"width": 68,
"id": 11,
"position": {
"row_max": 462,
"column_max": 88,
"column_min": 20,
"row_min": 394
}
},
{
"height": 12,
"class": "Compo",
"width": 11,
"id": 12,
"position": {
"row_max": 449,
"column_max": 161,
"column_min": 150,
"row_min": 437
}
},
{
"height": 69,
"class": "Compo",
"width": 68,
"id": 13,
"position": {
"row_max": 540,
"column_max": 88,
"column_min": 20,
"row_min": 471
}
},
{
"height": 13,
"class": "Compo",
"width": 15,
"id": 14,
"position": {
"row_max": 527,
"column_max": 108,
"column_min": 93,
"row_min": 514
}
},
{
"height": 13,
"class": "Compo",
"width": 11,
"id": 15,
"position": {
"row_max": 527,
"column_max": 158,
"column_min": 147,
"row_min": 514
}
},
{
"height": 67,
"class": "Compo",
"width": 68,
"id": 16,
"position": {
"row_max": 616,
"column_max": 88,
"column_min": 20,
"row_min": 549
}
},
{
"height": 12,
"class": "Compo",
"width": 11,
"id": 17,
"position": {
"row_max": 603,
"column_max": 158,
"column_min": 147,
"row_min": 591
}
},
{
"height": 67,
"class": "Compo",
"width": 68,
"id": 18,
"position": {
"row_max": 693,
"column_max": 88,
"column_min": 20,
"row_min": 626
}
},
{
"height": 11,
"class": "Compo",
"width": 15,
"id": 19,
"position": {
"row_max": 680,
"column_max": 108,
"column_min": 93,
"row_min": 669
}
},
{
"height": 12,
"class": "Compo",
"width": 11,
"id": 20,
"position": {
"row_max": 680,
"column_max": 161,
"column_min": 150,
"row_min": 668
}
},
{
"height": 16,
"class": "Compo",
"width": 66,
"id": 21,
"position": {
"row_max": 720,
"column_max": 87,
"column_min": 21,
"row_min": 704
}
},
{
"height": 18,
"text_content": "X X Cancel",
"id": 22,
"class": "Text",
"width": 94,
"position": {
"row_max": 62,
"column_max": 437,
"column_min": 343,
"row_min": 44
}
},
{
"height": 15,
"text_content": "Stuff You Should Know",
"id": 23,
"class": "Text",
"width": 161,
"position": {
"row_max": 114,
"column_max": 255,
"column_min": 94,
"row_min": 99
}
},
{
"height": 13,
"text_content": "+ 26.7k",
"id": 24,
"class": "Text",
"width": 32,
"position": {
"row_max": 140,
"column_max": 136,
"column_min": 104,
"row_min": 127
}
},
{
"height": 12,
"text_content": "665.Ok",
"id": 25,
"class": "Text",
"width": 30,
"position": {
"row_max": 139,
"column_max": 201,
"column_min": 171,
"row_min": 127
}
},
{
"height": 19,
"text_content": "Stuff You Missed in History Class",
"id": 26,
"class": "Text",
"width": 229,
"position": {
"row_max": 195,
"column_max": 323,
"column_min": 94,
"row_min": 176
}
},
{
"height": 13,
"text_content": "& + 13.7k",
"id": 27,
"class": "Text",
"width": 42,
"position": {
"row_max": 217,
"column_max": 136,
"column_min": 94,
"row_min": 204
}
},
{
"height": 12,
"text_content": "274.Ok",
"id": 28,
"class": "Text",
"width": 30,
"position": {
"row_max": 216,
"column_max": 201,
"column_min": 171,
"row_min": 204
}
},
{
"height": 15,
"text_content": "Stuff To Blow Your Mind",
"id": 29,
"class": "Text",
"width": 170,
"position": {
"row_max": 269,
"column_max": 264,
"column_min": 94,
"row_min": 254
}
},
{
"height": 13,
"text_content": "+ 12.7k",
"id": 30,
"class": "Text",
"width": 31,
"position": {
"row_max": 294,
"column_max": 136,
"column_min": 105,
"row_min": 281
}
},
{
"height": 12,
"text_content": "190.4k",
"id": 31,
"class": "Text",
"width": 29,
"position": {
"row_max": 294,
"column_max": 201,
"column_min": 172,
"row_min": 282
}
},
{
"height": 19,
"text_content": "Stuff They Don't Want You To Know Audio",
"id": 32,
"class": "Text",
"width": 296,
"position": {
"row_max": 349,
"column_max": 390,
"column_min": 94,
"row_min": 330
}
},
{
"height": 12,
"text_content": "+ 3.4k",
"id": 33,
"class": "Text",
"width": 26,
"position": {
"row_max": 371,
"column_max": 130,
"column_min": 104,
"row_min": 359
}
},
{
"height": 12,
"text_content": "72.2k",
"id": 34,
"class": "Text",
"width": 25,
"position": {
"row_max": 371,
"column_max": 190,
"column_min": 165,
"row_min": 359
}
},
{
"height": 16,
"text_content": "Stuff Mom Never Told You",
"id": 35,
"class": "Text",
"width": 185,
"position": {
"row_max": 424,
"column_max": 279,
"column_min": 94,
"row_min": 408
}
},
{
"height": 12,
"text_content": "2+ 2.Ok",
"id": 36,
"class": "Text",
"width": 37,
"position": {
"row_max": 448,
"column_max": 130,
"column_min": 93,
"row_min": 436
}
},
{
"height": 12,
"text_content": "48.5k",
"id": 37,
"class": "Text",
"width": 25,
"position": {
"row_max": 448,
"column_max": 190,
"column_min": 165,
"row_min": 436
}
},
{
"height": 17,
"text_content": "The Purple Stuff Podcast",
"id": 38,
"class": "Text",
"width": 175,
"position": {
"row_max": 502,
"column_max": 267,
"column_min": 92,
"row_min": 485
}
},
{
"height": 13,
"text_content": "+ 949",
"id": 39,
"class": "Text",
"width": 23,
"position": {
"row_max": 526,
"column_max": 128,
"column_min": 105,
"row_min": 513
}
},
{
"height": 12,
"text_content": "625.3k",
"id": 40,
"class": "Text",
"width": 29,
"position": {
"row_max": 525,
"column_max": 192,
"column_min": 163,
"row_min": 513
}
},
{
"height": 14,
"text_content": "Catholic Stuff You Should Know",
"id": 41,
"class": "Text",
"width": 222,
"position": {
"row_max": 577,
"column_max": 316,
"column_min": 94,
"row_min": 563
}
},
{
"height": 12,
"text_content": "2+ 683",
"id": 42,
"class": "Text",
"width": 34,
"position": {
"row_max": 602,
"column_max": 128,
"column_min": 94,
"row_min": 590
}
},
{
"height": 12,
"text_content": "8.8k",
"id": 43,
"class": "Text",
"width": 18,
"position": {
"row_max": 602,
"column_max": 181,
"column_min": 163,
"row_min": 590
}
},
{
"height": 18,
"text_content": "Stuff They Don't Want You To Know",
"id": 44,
"class": "Text",
"width": 251,
"position": {
"row_max": 657,
"column_max": 345,
"column_min": 94,
"row_min": 639
}
},
{
"height": 12,
"text_content": "+ 1.1k",
"id": 45,
"class": "Text",
"width": 26,
"position": {
"row_max": 679,
"column_max": 130,
"column_min": 104,
"row_min": 667
}
},
{
"height": 12,
"text_content": "5.4k",
"id": 46,
"class": "Text",
"width": 18,
"position": {
"row_max": 679,
"column_max": 184,
"column_min": 166,
"row_min": 667
}
}
],
"img_shape": [
800,
450,
3
]
}

239
UIED/data/output/ocr/1565.json Normal file
View File

@@ -0,0 +1,239 @@
{
"img_shape": [
835,
521,
3
],
"texts": [
{
"content": "ALL",
"column_min": 101,
"row_max": 48,
"height": 22,
"column_max": 130,
"width": 29,
"id": 0,
"row_min": 26
},
{
"content": "ASK AC",
"column_min": 19,
"row_max": 241,
"height": 20,
"column_max": 78,
"width": 59,
"id": 1,
"row_min": 221
},
{
"content": "Do really need",
"column_min": 19,
"row_max": 281,
"height": 36,
"column_max": 208,
"width": 189,
"id": 2,
"row_min": 245
},
{
"content": "a \u2022 mesh 00000",
"column_min": 218,
"row_max": 292,
"height": 47,
"column_max": 306,
"width": 88,
"id": 3,
"row_min": 245
},
{
"content": "network ?",
"column_min": 316,
"row_max": 280,
"height": 37,
"column_max": 429,
"width": 113,
"id": 4,
"row_min": 243
},
{
"content": "Microsoft thinks people want",
"column_min": 143,
"row_max": 344,
"height": 26,
"column_max": 434,
"width": 291,
"id": 5,
"row_min": 318
},
{
"content": "ultra portable headaches",
"column_min": 141,
"row_max": 373,
"height": 30,
"column_max": 390,
"width": 249,
"id": 6,
"row_min": 343
},
{
"content": "Jerry Hildenbrand",
"column_min": 144,
"row_max": 399,
"height": 20,
"column_max": 282,
"width": 138,
"id": 7,
"row_min": 379
},
{
"content": "COMMENTS 57m",
"column_min": 310,
"row_max": 399,
"height": 20,
"column_max": 455,
"width": 145,
"id": 8,
"row_min": 379
},
{
"content": "My Disney Experience is a",
"column_min": 143,
"row_max": 491,
"height": 28,
"column_max": 400,
"width": 257,
"id": 9,
"row_min": 463
},
{
"content": "whole new experience in new",
"column_min": 142,
"row_max": 515,
"height": 25,
"column_max": 434,
"width": 292,
"id": 10,
"row_min": 490
},
{
"content": "update",
"column_min": 142,
"row_max": 541,
"height": 25,
"column_max": 210,
"width": 68,
"id": 11,
"row_min": 516
},
{
"content": "Ara Wagoner",
"column_min": 142,
"row_max": 571,
"height": 23,
"column_max": 242,
"width": 100,
"id": 12,
"row_min": 548
},
{
"content": "COMMENTS 3h",
"column_min": 271,
"row_max": 571,
"height": 23,
"column_max": 400,
"width": 129,
"id": 13,
"row_min": 548
},
{
"content": "The best",
"column_min": 142,
"row_max": 632,
"height": 28,
"column_max": 228,
"width": 86,
"id": 14,
"row_min": 604
},
{
"content": "games VR for Gear",
"column_min": 236,
"row_max": 658,
"height": 53,
"column_max": 337,
"width": 101,
"id": 15,
"row_min": 605
},
{
"content": "your",
"column_min": 344,
"row_max": 634,
"height": 28,
"column_max": 387,
"width": 43,
"id": 16,
"row_min": 606
},
{
"content": "Samsung",
"column_min": 141,
"row_max": 660,
"height": 28,
"column_max": 234,
"width": 93,
"id": 17,
"row_min": 632
},
{
"content": "Russell Holly",
"column_min": 142,
"row_max": 689,
"height": 24,
"column_max": 241,
"width": 99,
"id": 18,
"row_min": 665
},
{
"content": "COMMENTS 4h",
"column_min": 271,
"row_max": 689,
"height": 24,
"column_max": 399,
"width": 128,
"id": 19,
"row_min": 665
},
{
"content": "Here's how to get a little more",
"column_min": 143,
"row_max": 775,
"height": 26,
"column_max": 440,
"width": 297,
"id": 20,
"row_min": 749
},
{
"content": "Android Central in your life !",
"column_min": 142,
"row_max": 802,
"height": 26,
"column_max": 413,
"width": 271,
"id": 21,
"row_min": 776
},
{
"content": "Florence lon5 COMMENTS 5h",
"column_min": 142,
"row_max": 831,
"height": 21,
"column_max": 396,
"width": 254,
"id": 22,
"row_min": 810
}
]
}

BIN
UIED/data/output/ocr/1565.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 446 KiB

489
UIED/data/output/ocr/497.json Normal file
View File

@@ -0,0 +1,489 @@
{
"texts": [
{
"height": 12,
"row_max": 23,
"content": "FAK",
"id": 0,
"width": 31,
"column_max": 373,
"row_min": 11,
"column_min": 342
},
{
"height": 41,
"row_max": 51,
"content": "fo",
"id": 1,
"width": 96,
"column_max": 188,
"row_min": 10,
"column_min": 92
},
{
"height": 31,
"row_max": 47,
"content": "3:17",
"id": 2,
"width": 68,
"column_max": 1055,
"row_min": 16,
"column_min": 987
},
{
"height": 33,
"row_max": 143,
"content": "stuff",
"id": 3,
"width": 77,
"column_max": 249,
"row_min": 110,
"column_min": 172
},
{
"height": 45,
"row_max": 151,
"content": "X X Cancel",
"id": 4,
"width": 225,
"column_max": 1049,
"row_min": 106,
"column_min": 824
},
{
"height": 36,
"row_max": 275,
"content": "Stuff You Should Know",
"id": 5,
"width": 387,
"column_max": 613,
"row_min": 239,
"column_min": 226
},
{
"height": 18,
"row_max": 272,
"content": "STUFF",
"id": 6,
"width": 54,
"column_max": 119,
"row_min": 254,
"column_min": 65
},
{
"height": 17,
"row_max": 291,
"content": "YOU SHOULD",
"id": 7,
"width": 127,
"column_max": 193,
"row_min": 274,
"column_min": 66
},
{
"height": 18,
"row_max": 311,
"content": "KNOW",
"id": 8,
"width": 65,
"column_max": 132,
"row_min": 293,
"column_min": 67
},
{
"height": 30,
"row_max": 336,
"content": "+ 26.7k",
"id": 9,
"width": 77,
"column_max": 327,
"row_min": 306,
"column_min": 250
},
{
"height": 29,
"row_max": 335,
"content": "665.Ok",
"id": 10,
"width": 71,
"column_max": 483,
"row_min": 306,
"column_min": 412
},
{
"height": 16,
"row_max": 343,
"content": "PODCAST",
"id": 11,
"width": 80,
"column_max": 146,
"row_min": 327,
"column_min": 66
},
{
"height": 47,
"row_max": 470,
"content": "Stuff You Missed in History Class",
"id": 12,
"width": 550,
"column_max": 776,
"row_min": 423,
"column_min": 226
},
{
"height": 18,
"row_max": 482,
"content": "STUFF",
"id": 13,
"width": 32,
"column_max": 186,
"row_min": 464,
"column_min": 154
},
{
"height": 31,
"row_max": 522,
"content": "& + 13.7k",
"id": 14,
"width": 101,
"column_max": 328,
"row_min": 491,
"column_min": 227
},
{
"height": 29,
"row_max": 520,
"content": "274.Ok",
"id": 15,
"width": 71,
"column_max": 483,
"row_min": 491,
"column_min": 412
},
{
"height": 12,
"row_max": 520,
"content": "YOU MISSED IN",
"id": 16,
"width": 79,
"column_max": 141,
"row_min": 508,
"column_min": 62
},
{
"height": 20,
"row_max": 539,
"content": "HISTORY CLASS",
"id": 17,
"width": 143,
"column_max": 204,
"row_min": 519,
"column_min": 61
},
{
"height": 43,
"row_max": 654,
"content": "BLOW stuff YOUR to",
"id": 18,
"width": 79,
"column_max": 198,
"row_min": 611,
"column_min": 119
},
{
"height": 35,
"row_max": 646,
"content": "Stuff To Blow Your Mind",
"id": 19,
"width": 408,
"column_max": 634,
"row_min": 611,
"column_min": 226
},
{
"height": 32,
"row_max": 686,
"content": "MIND",
"id": 20,
"width": 98,
"column_max": 196,
"row_min": 654,
"column_min": 98
},
{
"height": 30,
"row_max": 706,
"content": "+ 12.7k",
"id": 21,
"width": 75,
"column_max": 327,
"row_min": 676,
"column_min": 252
},
{
"height": 29,
"row_max": 707,
"content": "190.4k",
"id": 22,
"width": 70,
"column_max": 483,
"row_min": 678,
"column_min": 413
},
{
"height": 68,
"row_max": 869,
"content": "AUDIO STUFF DON THEY WANT KNOW YOU TO",
"id": 23,
"width": 94,
"column_max": 187,
"row_min": 801,
"column_min": 93
},
{
"height": 44,
"row_max": 838,
"content": "Stuff They Don't Want You To Know Audio",
"id": 24,
"width": 711,
"column_max": 937,
"row_min": 794,
"column_min": 226
},
{
"height": 29,
"row_max": 891,
"content": "+ 3.4k",
"id": 25,
"width": 63,
"column_max": 313,
"row_min": 862,
"column_min": 250
},
{
"height": 29,
"row_max": 892,
"content": "72.2k",
"id": 26,
"width": 58,
"column_max": 456,
"row_min": 863,
"column_min": 398
},
{
"height": 13,
"row_max": 1003,
"content": "stuff mom",
"id": 27,
"width": 75,
"column_max": 157,
"row_min": 990,
"column_min": 82
},
{
"height": 55,
"row_max": 1056,
"content": "never told you",
"id": 28,
"width": 133,
"column_max": 198,
"row_min": 1001,
"column_min": 65
},
{
"height": 38,
"row_max": 1019,
"content": "Stuff Mom Never Told You",
"id": 29,
"width": 445,
"column_max": 671,
"row_min": 981,
"column_min": 226
},
{
"height": 18,
"row_max": 1072,
"content": "audio",
"id": 30,
"width": 44,
"column_max": 191,
"row_min": 1054,
"column_min": 147
},
{
"height": 29,
"row_max": 1076,
"content": "2+ 2.Ok",
"id": 31,
"width": 88,
"column_max": 313,
"row_min": 1047,
"column_min": 225
},
{
"height": 28,
"row_max": 1076,
"content": "48.5k",
"id": 32,
"width": 58,
"column_max": 456,
"row_min": 1048,
"column_min": 398
},
{
"height": 44,
"row_max": 1182,
"content": "PURPLE PODCAST STUFF",
"id": 33,
"width": 137,
"column_max": 204,
"row_min": 1138,
"column_min": 67
},
{
"height": 42,
"row_max": 1206,
"content": "The Purple Stuff Podcast",
"id": 34,
"width": 418,
"column_max": 641,
"row_min": 1164,
"column_min": 223
},
{
"height": 31,
"row_max": 1263,
"content": "+ 949",
"id": 35,
"width": 56,
"column_max": 308,
"row_min": 1232,
"column_min": 252
},
{
"height": 28,
"row_max": 1261,
"content": "625.3k",
"id": 36,
"width": 70,
"column_max": 463,
"row_min": 1233,
"column_min": 393
},
{
"height": 154,
"row_max": 1470,
"content": "CATHO . YOU SHOULO KNOW STUFF .",
"id": 37,
"width": 159,
"column_max": 206,
"row_min": 1316,
"column_min": 47
},
{
"height": 33,
"row_max": 1385,
"content": "Catholic Stuff You Should Know",
"id": 38,
"width": 533,
"column_max": 759,
"row_min": 1352,
"column_min": 226
},
{
"height": 28,
"row_max": 1446,
"content": "2+ 683",
"id": 39,
"width": 81,
"column_max": 308,
"row_min": 1418,
"column_min": 227
},
{
"height": 28,
"row_max": 1446,
"content": "8.8k",
"id": 40,
"width": 43,
"column_max": 436,
"row_min": 1418,
"column_min": 393
},
{
"height": 14,
"row_max": 1550,
"content": "STUFE",
"id": 41,
"width": 42,
"column_max": 152,
"row_min": 1536,
"column_min": 110
},
{
"height": 45,
"row_max": 1593,
"content": "THEY DONT WANT YOU TO KNOW",
"id": 42,
"width": 75,
"column_max": 166,
"row_min": 1548,
"column_min": 91
},
{
"height": 45,
"row_max": 1579,
"content": "Stuff They Don't Want You To Know",
"id": 43,
"width": 604,
"column_max": 830,
"row_min": 1534,
"column_min": 226
},
{
"height": 29,
"row_max": 1631,
"content": "+ 1.1k",
"id": 44,
"width": 63,
"column_max": 314,
"row_min": 1602,
"column_min": 251
},
{
"height": 28,
"row_max": 1631,
"content": "5.4k",
"id": 45,
"width": 43,
"column_max": 442,
"row_min": 1603,
"column_min": 399
},
{
"height": 22,
"row_max": 1660,
"content": "howstuffworks.com",
"id": 46,
"width": 143,
"column_max": 204,
"row_min": 1638,
"column_min": 61
},
{
"height": 16,
"row_max": 1724,
"content": "THE",
"id": 47,
"width": 38,
"column_max": 152,
"row_min": 1708,
"column_min": 114
}
],
"img_shape": [
1920,
1080,
3
]
}

BIN
UIED/data/output/ocr/497.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 901 KiB

BIN
UIED/detect_compo/__pycache__/ip_region_proposal.cpython-312.pyc Normal file
View File

Binary file not shown.

BIN
UIED/detect_compo/__pycache__/ip_region_proposal.cpython-35.pyc Normal file
View File

Binary file not shown.

56
UIED/detect_compo/deprecated/Block.py Normal file
View File

@@ -0,0 +1,56 @@
import cv2
from os.path import join as pjoin
import time
import numpy as np
from detect_compo.lib_ip.Component import Component
from config.CONFIG_UIED import Config
C = Config()
class Block(Component):
def __init__(self, region, image_shape):
super().__init__(region, image_shape)
self.category = 'Block'
self.parent = None
self.children = []
self.uicompo_ = None
self.top_or_botm = None
self.redundant = False
def block_is_uicompo(self, image_shape, max_compo_scale):
'''
Check the if the block is a ui component according to its relative size
'''
row, column = image_shape[:2]
# print(height, height / row, max_compo_scale[0], height / row > max_compo_scale[0])
# draw.draw_bounding_box(org, [corner], show=True)
# ignore atomic components
if self.bbox.height / row > max_compo_scale[0] or self.bbox.width / column > max_compo_scale[1]:
return False
return True
def block_is_top_or_bottom_bar(self, image_shape, top_bottom_height):
'''
Check if the block is top bar or bottom bar
'''
height, width = image_shape[:2]
(column_min, row_min, column_max, row_max) = self.bbox.put_bbox()
if column_min < 5 and row_min < 5 and \
width - column_max < 5 and row_max < height * top_bottom_height[0]:
self.uicompo_ = True
return True
if column_min < 5 and row_min > height * top_bottom_height[1] and \
width - column_max < 5 and height - row_max < 5:
self.uicompo_ = True
return True
return False
def block_erase_from_bin(self, binary, pad):
(column_min, row_min, column_max, row_max) = self.put_bbox()
column_min = max(column_min - pad, 0)
column_max = min(column_max + pad, binary.shape[1])
row_min = max(row_min - pad, 0)
row_max = min(row_max + pad, binary.shape[0])
cv2.rectangle(binary, (column_min, row_min), (column_max, row_max), (0), -1)

108
UIED/detect_compo/deprecated/block_division.py Normal file
View File

@@ -0,0 +1,108 @@
import cv2
import numpy as np
from random import randint as rint
import time
import detect_compo.lib_ip.ip_preprocessing as pre
import detect_compo.lib_ip.ip_detection as det
import detect_compo.lib_ip.ip_draw as draw
import detect_compo.lib_ip.ip_segment as seg
from detect_compo.lib_ip.Block import Block
from config.CONFIG_UIED import Config
C = Config()
def block_hierarchy(blocks):
for i in range(len(blocks) - 1):
for j in range(i + 1, len(blocks)):
relation = blocks[i].compo_relation(blocks[j])
if relation == -1:
blocks[j].children.append(i)
if relation == 1:
blocks[i].children.append(j)
return
def block_bin_erase_all_blk(binary, blocks, pad=0, show=False):
'''
erase the block parts from the binary map
:param binary: binary map of original image
:param blocks_corner: corners of detected layout block
:param show: show or not
:param pad: expand the bounding boxes of blocks
:return: binary map without block parts
'''
bin_org = binary.copy()
for block in blocks:
block.block_erase_from_bin(binary, pad)
if show:
cv2.imshow('before', bin_org)
cv2.imshow('after', binary)
cv2.waitKey()
def block_division(grey, org, grad_thresh,
show=False, write_path=None,
step_h=10, step_v=10,
line_thickness=C.THRESHOLD_LINE_THICKNESS,
min_rec_evenness=C.THRESHOLD_REC_MIN_EVENNESS,
max_dent_ratio=C.THRESHOLD_REC_MAX_DENT_RATIO,
min_block_height_ratio=C.THRESHOLD_BLOCK_MIN_HEIGHT):
'''
:param grey: grey-scale of original image
:return: corners: list of [(top_left, bottom_right)]
-> top_left: (column_min, row_min)
-> bottom_right: (column_max, row_max)
'''
blocks = []
mask = np.zeros((grey.shape[0]+2, grey.shape[1]+2), dtype=np.uint8)
broad = np.zeros((grey.shape[0], grey.shape[1], 3), dtype=np.uint8)
broad_all = broad.copy()
row, column = grey.shape[0], grey.shape[1]
for x in range(0, row, step_h):
for y in range(0, column, step_v):
if mask[x, y] == 0:
# region = flood_fill_bfs(grey, x, y, mask)
# flood fill algorithm to get background (layout block)
mask_copy = mask.copy()
ff = cv2.floodFill(grey, mask, (y, x), None, grad_thresh, grad_thresh, cv2.FLOODFILL_MASK_ONLY)
# ignore small regions
if ff[0] < 500: continue
mask_copy = mask - mask_copy
region = np.reshape(cv2.findNonZero(mask_copy[1:-1, 1:-1]), (-1, 2))
region = [(p[1], p[0]) for p in region]
block = Block(region, grey.shape)
# draw.draw_region(region, broad_all)
# if block.height < 40 and block.width < 40:
# continue
if block.height < 30:
continue
# print(block.area / (row * column))
if block.area / (row * column) > 0.9:
continue
elif block.area / (row * column) > 0.7:
block.redundant = True
# get the boundary of this region
# ignore lines
if block.compo_is_line(line_thickness):
continue
# ignore non-rectangle as blocks must be rectangular
if not block.compo_is_rectangle(min_rec_evenness, max_dent_ratio):
continue
# if block.height/row < min_block_height_ratio:
# continue
blocks.append(block)
# draw.draw_region(region, broad)
if show:
cv2.imshow('flood-fill all', broad_all)
cv2.imshow('block', broad)
cv2.waitKey()
if write_path is not None:
cv2.imwrite(write_path, broad)
return blocks

461
UIED/detect_compo/deprecated/ip_detection_utils.py Normal file
View File

@@ -0,0 +1,461 @@
import numpy as np
import cv2
from collections import Counter
import lib_ip.ip_draw as draw
from config.CONFIG_UIED import Config
C = Config()
# detect object(connected region)
# def boundary_bfs_connected_area(img, x, y, mark):
# def neighbor(img, x, y, mark, stack):
# for i in range(x - 1, x + 2):
# if i < 0 or i >= img.shape[0]: continue
# for j in range(y - 1, y + 2):
# if j < 0 or j >= img.shape[1]: continue
# if img[i, j] == 255 and mark[i, j] == 0:
# stack.append([i, j])
# mark[i, j] = 255
#
# stack = [[x, y]] # points waiting for inspection
# area = [[x, y]] # points of this area
# mark[x, y] = 255 # drawing broad
#
# while len(stack) > 0:
# point = stack.pop()
# area.append(point)
# neighbor(img, point[0], point[1], mark, stack)
# return area
# def line_check_perpendicular(lines_h, lines_v, max_thickness):
# """
# lines: [line_h, line_v]
# -> line_h: horizontal {'head':(column_min, row), 'end':(column_max, row), 'thickness':int)
# -> line_v: vertical {'head':(column, row_min), 'end':(column, row_max), 'thickness':int}
# """
# is_per_h = np.full(len(lines_h), False)
# is_per_v = np.full(len(lines_v), False)
# for i in range(len(lines_h)):
# # save the intersection point of h
# lines_h[i]['inter_point'] = set()
# h = lines_h[i]
#
# for j in range(len(lines_v)):
# # save the intersection point of v
# if 'inter_point' not in lines_v[j]: lines_v[j]['inter_point'] = set()
# v = lines_v[j]
#
# # if h is perpendicular to v in head of v
# if abs(h['head'][1]-v['head'][1]) <= max_thickness:
# if abs(h['head'][0] - v['head'][0]) <= max_thickness:
# lines_h[i]['inter_point'].add('head')
# lines_v[j]['inter_point'].add('head')
# is_per_h[i] = True
# is_per_v[j] = True
# elif abs(h['end'][0] - v['head'][0]) <= max_thickness:
# lines_h[i]['inter_point'].add('end')
# lines_v[j]['inter_point'].add('head')
# is_per_h[i] = True
# is_per_v[j] = True
#
# # if h is perpendicular to v in end of v
# elif abs(h['head'][1]-v['end'][1]) <= max_thickness:
# if abs(h['head'][0] - v['head'][0]) <= max_thickness:
# lines_h[i]['inter_point'].add('head')
# lines_v[j]['inter_point'].add('end')
# is_per_h[i] = True
# is_per_v[j] = True
# elif abs(h['end'][0] - v['head'][0]) <= max_thickness:
# lines_h[i]['inter_point'].add('end')
# lines_v[j]['inter_point'].add('end')
# is_per_h[i] = True
# is_per_v[j] = True
# per_h = []
# per_v = []
# for i in range(len(is_per_h)):
# if is_per_h[i]:
# lines_h[i]['inter_point'] = list(lines_h[i]['inter_point'])
# per_h.append(lines_h[i])
# for i in range(len(is_per_v)):
# if is_per_v[i]:
# lines_v[i]['inter_point'] = list(lines_v[i]['inter_point'])
# per_v.append(lines_v[i])
# return per_h, per_v
# def line_shrink_corners(corner, lines_h, lines_v):
# """
# shrink the corner according to lines:
# col_min_shrink: shrink right (increase)
# col_max_shrink: shrink left (decrease)
# row_min_shrink: shrink down (increase)
# row_max_shrink: shrink up (decrease)
# :param lines_h: horizontal {'head':(column_min, row), 'end':(column_max, row), 'thickness':int)
# :param lines_v: vertical {'head':(column, row_min), 'end':(column, row_max), 'thickness':int}
# :return: shrunken corner: (top_left, bottom_right)
# """
# (col_min, row_min), (col_max, row_max) = corner
# col_min_shrink, row_min_shrink = col_min, row_min
# col_max_shrink, row_max_shrink = col_max, row_max
# valid_frame = False
#
# for h in lines_h:
# # ignore outer border
# if len(h['inter_point']) == 2:
# valid_frame = True
# continue
# # shrink right -> col_min move to end
# if h['inter_point'][0] == 'head':
# col_min_shrink = max(h['end'][0], col_min_shrink)
# # shrink left -> col_max move to head
# elif h['inter_point'][0] == 'end':
# col_max_shrink = min(h['head'][0], col_max_shrink)
#
# for v in lines_v:
# # ignore outer border
# if len(v['inter_point']) == 2:
# valid_frame = True
# continue
# # shrink down -> row_min move to end
# if v['inter_point'][0] == 'head':
# row_min_shrink = max(v['end'][1], row_min_shrink)
# # shrink up -> row_max move to head
# elif v['inter_point'][0] == 'end':
# row_max_shrink = min(v['head'][1], row_max_shrink)
#
# # return the shrunken corner if only there is line intersecting with two other lines
# if valid_frame:
# return (col_min_shrink, row_min_shrink), (col_max_shrink, row_max_shrink)
# return corner
# def line_cvt_relative_position(col_min, row_min, lines_h, lines_v):
# """
# convert the relative position of lines in the entire image
# :param col_min: based column the img lines belong to
# :param row_min: based row the img lines belong to
# :param lines_h: horizontal {'head':(column_min, row), 'end':(column_max, row), 'thickness':int)
# :param lines_v: vertical {'head':(column, row_min), 'end':(column, row_max), 'thickness':int}
# :return: lines_h_cvt, lines_v_cvt
# """
# for h in lines_h:
# h['head'][0] += col_min
# h['head'][1] += row_min
# h['end'][0] += col_min
# h['end'][1] += row_min
# for v in lines_v:
# v['head'][0] += col_min
# v['head'][1] += row_min
# v['end'][0] += col_min
# v['end'][1] += row_min
#
# return lines_h, lines_v
# check if an object is so slim
# @boundary: [border_up, border_bottom, border_left, border_right]
# -> up, bottom: (column_index, min/max row border)
# -> left, right: (row_index, min/max column border) detect range of each row
def clipping_by_line(boundary, boundary_rec, lines):
boundary = boundary.copy()
for orient in lines:
# horizontal
if orient == 'h':
# column range of sub area
r1, r2 = 0, 0
for line in lines[orient]:
if line[0] == 0:
r1 = line[1]
continue
r2 = line[0]
b_top = []
b_bottom = []
for i in range(len(boundary[0])):
if r2 > boundary[0][i][0] >= r1:
b_top.append(boundary[0][i])
for i in range(len(boundary[1])):
if r2 > boundary[1][i][0] >= r1:
b_bottom.append(boundary[1][i])
b_left = [x for x in boundary[2]] # (row_index, min column border)
for i in range(len(b_left)):
if b_left[i][1] < r1:
b_left[i][1] = r1
b_right = [x for x in boundary[3]] # (row_index, max column border)
for i in range(len(b_right)):
if b_right[i][1] > r2:
b_right[i][1] = r2
boundary_rec.append([b_top, b_bottom, b_left, b_right])
r1 = line[1]
# remove imgs that contain text
# def rm_text(org, corners, compo_class,
# max_text_height=C.THRESHOLD_TEXT_MAX_HEIGHT, max_text_width=C.THRESHOLD_TEXT_MAX_WIDTH,
# ocr_padding=C.OCR_PADDING, ocr_min_word_area=C.OCR_MIN_WORD_AREA, show=False):
# """
# Remove area that full of text
# :param org: original image
# :param corners: [(top_left, bottom_right)]
# -> top_left: (column_min, row_min)
# -> bottom_right: (column_max, row_max)
# :param compo_class: classes of corners
# :param max_text_height: Too large to be text
# :param max_text_width: Too large to be text
# :param ocr_padding: Padding for clipping
# :param ocr_min_word_area: If too text area ratio is too large
# :param show: Show or not
# :return: corners without text objects
# """
# new_corners = []
# new_class = []
# for i in range(len(corners)):
# corner = corners[i]
# (top_left, bottom_right) = corner
# (col_min, row_min) = top_left
# (col_max, row_max) = bottom_right
# height = row_max - row_min
# width = col_max - col_min
# # highly likely to be block or img if too large
# if height > max_text_height and width > max_text_width:
# new_corners.append(corner)
# new_class.append(compo_class[i])
# else:
# row_min = row_min - ocr_padding if row_min - ocr_padding >= 0 else 0
# row_max = row_max + ocr_padding if row_max + ocr_padding < org.shape[0] else org.shape[0]
# col_min = col_min - ocr_padding if col_min - ocr_padding >= 0 else 0
# col_max = col_max + ocr_padding if col_max + ocr_padding < org.shape[1] else org.shape[1]
# # check if this area is text
# clip = org[row_min: row_max, col_min: col_max]
# if not ocr.is_text(clip, ocr_min_word_area, show=show):
# new_corners.append(corner)
# new_class.append(compo_class[i])
# return new_corners, new_class
# def rm_img_in_compo(corners_img, corners_compo):
# """
# Remove imgs in component
# """
# corners_img_new = []
# for img in corners_img:
# is_nested = False
# for compo in corners_compo:
# if util.corner_relation(img, compo) == -1:
# is_nested = True
# break
# if not is_nested:
# corners_img_new.append(img)
# return corners_img_new
# def block_or_compo(org, binary, corners,
# max_thickness=C.THRESHOLD_BLOCK_MAX_BORDER_THICKNESS, max_block_cross_points=C.THRESHOLD_BLOCK_MAX_CROSS_POINT,
# min_compo_w_h_ratio=C.THRESHOLD_UICOMPO_MIN_W_H_RATIO, max_compo_w_h_ratio=C.THRESHOLD_UICOMPO_MAX_W_H_RATIO,
# min_block_edge=C.THRESHOLD_BLOCK_MIN_EDGE_LENGTH):
# """
# Check if the objects are img components or just block
# :param org: Original image
# :param binary: Binary image from pre-processing
# :param corners: [(top_left, bottom_right)]
# -> top_left: (column_min, row_min)
# -> bottom_right: (column_max, row_max)
# :param max_thickness: The max thickness of border of blocks
# :param max_block_cross_points: Ratio of point of interaction
# :return: corners of blocks and imgs
# """
# blocks = []
# imgs = []
# compos = []
# for corner in corners:
# (top_left, bottom_right) = corner
# (col_min, row_min) = top_left
# (col_max, row_max) = bottom_right
# height = row_max - row_min
# width = col_max - col_min
#
# block = False
# vacancy = [0, 0, 0, 0]
# for i in range(1, max_thickness):
# try:
# # top to bottom
# if vacancy[0] == 0 and (col_max - col_min - 2 * i) is not 0 and (
# np.sum(binary[row_min + i, col_min + i: col_max - i]) / 255) / (col_max - col_min - 2 * i) <= max_block_cross_points:
# vacancy[0] = 1
# # bottom to top
# if vacancy[1] == 0 and (col_max - col_min - 2 * i) is not 0 and (
# np.sum(binary[row_max - i, col_min + i: col_max - i]) / 255) / (col_max - col_min - 2 * i) <= max_block_cross_points:
# vacancy[1] = 1
# # left to right
# if vacancy[2] == 0 and (row_max - row_min - 2 * i) is not 0 and (
# np.sum(binary[row_min + i: row_max - i, col_min + i]) / 255) / (row_max - row_min - 2 * i) <= max_block_cross_points:
# vacancy[2] = 1
# # right to left
# if vacancy[3] == 0 and (row_max - row_min - 2 * i) is not 0 and (
# np.sum(binary[row_min + i: row_max - i, col_max - i]) / 255) / (row_max - row_min - 2 * i) <= max_block_cross_points:
# vacancy[3] = 1
# if np.sum(vacancy) == 4:
# block = True
# except:
# pass
#
# # too big to be UI components
# if block:
# if height > min_block_edge and width > min_block_edge:
# blocks.append(corner)
# else:
# if min_compo_w_h_ratio < width / height < max_compo_w_h_ratio:
# compos.append(corner)
# # filter out small objects
# else:
# if height > min_block_edge:
# imgs.append(corner)
# else:
# if min_compo_w_h_ratio < width / height < max_compo_w_h_ratio:
# compos.append(corner)
# return blocks, imgs, compos
# def compo_on_img(processing, org, binary, clf,
# compos_corner, compos_class):
# """
# Detect potential UI components inner img;
# Only leave non-img
# """
# pad = 2
# for i in range(len(compos_corner)):
# if compos_class[i] != 'img':
# continue
# ((col_min, row_min), (col_max, row_max)) = compos_corner[i]
# col_min = max(col_min - pad, 0)
# col_max = min(col_max + pad, org.shape[1])
# row_min = max(row_min - pad, 0)
# row_max = min(row_max + pad, org.shape[0])
# area = (col_max - col_min) * (row_max - row_min)
# if area < 600:
# continue
#
# clip_org = org[row_min:row_max, col_min:col_max]
# clip_bin_inv = pre.reverse_binary(binary[row_min:row_max, col_min:col_max])
#
# compos_boundary_new, compos_corner_new, compos_class_new = processing(clip_org, clip_bin_inv, clf)
# compos_corner_new = util.corner_cvt_relative_position(compos_corner_new, col_min, row_min)
#
# assert len(compos_corner_new) == len(compos_class_new)
#
# # only leave non-img elements
# for i in range(len(compos_corner_new)):
# ((col_min_new, row_min_new), (col_max_new, row_max_new)) = compos_corner_new[i]
# area_new = (col_max_new - col_min_new) * (row_max_new - row_min_new)
# if compos_class_new[i] != 'img' and area_new / area < 0.8:
# compos_corner.append(compos_corner_new[i])
# compos_class.append(compos_class_new[i])
#
# return compos_corner, compos_class
# def strip_img(corners_compo, compos_class, corners_img):
# """
# Separate img from other compos
# :return: compos without img
# """
# corners_compo_withuot_img = []
# compo_class_withuot_img = []
# for i in range(len(compos_class)):
# if compos_class[i] == 'img':
# corners_img.append(corners_compo[i])
# else:
# corners_compo_withuot_img.append(corners_compo[i])
# compo_class_withuot_img.append(compos_class[i])
# return corners_compo_withuot_img, compo_class_withuot_img
# def merge_corner(corners, compos_class, min_selected_IoU=C.THRESHOLD_MIN_IOU, is_merge_nested_same=True):
# """
# Calculate the Intersection over Overlap (IoU) and merge corners according to the value of IoU
# :param is_merge_nested_same: if true, merge the nested corners with same class whatever the IoU is
# :param corners: corners: [(top_left, bottom_right)]
# -> top_left: (column_min, row_min)
# -> bottom_right: (column_max, row_max)
# :return: new corners
# """
# new_corners = []
# new_class = []
# for i in range(len(corners)):
# is_intersected = False
# for j in range(len(new_corners)):
# r = util.corner_relation_nms(corners[i], new_corners[j], min_selected_IoU)
# # r = util.corner_relation(corners[i], new_corners[j])
# if is_merge_nested_same:
# if compos_class[i] == new_class[j]:
# # if corners[i] is in new_corners[j], ignore corners[i]
# if r == -1:
# is_intersected = True
# break
# # if new_corners[j] is in corners[i], replace new_corners[j] with corners[i]
# elif r == 1:
# is_intersected = True
# new_corners[j] = corners[i]
#
# # if above IoU threshold, and corners[i] is in new_corners[j], ignore corners[i]
# if r == -2:
# is_intersected = True
# break
# # if above IoU threshold, and new_corners[j] is in corners[i], replace new_corners[j] with corners[i]
# elif r == 2:
# is_intersected = True
# new_corners[j] = corners[i]
# new_class[j] = compos_class[i]
#
# # containing and too small
# elif r == -3:
# is_intersected = True
# break
# elif r == 3:
# is_intersected = True
# new_corners[j] = corners[i]
#
# # if [i] and [j] are overlapped but no containing relation, merge corners when same class
# elif r == 4:
# is_intersected = True
# if compos_class[i] == new_class[j]:
# new_corners[j] = util.corner_merge_two_corners(corners[i], new_corners[j])
#
# if not is_intersected:
# new_corners.append(corners[i])
# new_class.append(compos_class[i])
# return new_corners, new_class
# def select_corner(corners, compos_class, class_name):
# """
# Select corners in given compo type
# """
# corners_wanted = []
# for i in range(len(compos_class)):
# if compos_class[i] == class_name:
# corners_wanted.append(corners[i])
# return corners_wanted
# def flood_fill_bfs(img, x_start, y_start, mark, grad_thresh):
# def neighbor(x, y):
# for i in range(x - 1, x + 2):
# if i < 0 or i >= img.shape[0]: continue
# for j in range(y - 1, y + 2):
# if j < 0 or j >= img.shape[1]: continue
# if mark[i, j] == 0 and abs(img[i, j] - img[x, y]) < grad_thresh:
# stack.append([i, j])
# mark[i, j] = 255
#
# stack = [[x_start, y_start]] # points waiting for inspection
# region = [[x_start, y_start]] # points of this connected region
# mark[x_start, y_start] = 255 # drawing broad
# while len(stack) > 0:
# point = stack.pop()
# region.append(point)
# neighbor(point[0], point[1])
# return region

123
UIED/detect_compo/deprecated/ip_segment.py Normal file
View File

@@ -0,0 +1,123 @@
import cv2
import numpy as np
import shutil
import os
from os.path import join as pjoin
def segment_img(org, segment_size, output_path, overlap=100):
if not os.path.exists(output_path):
os.mkdir(output_path)
height, width = np.shape(org)[0], np.shape(org)[1]
top = 0
bottom = segment_size
segment_no = 0
while top < height and bottom < height:
segment = org[top:bottom]
cv2.imwrite(os.path.join(output_path, str(segment_no) + '.png'), segment)
segment_no += 1
top += segment_size - overlap
bottom = bottom + segment_size - overlap if bottom + segment_size - overlap <= height else height
def clipping(img, components, pad=0, show=False):
"""
:param adjust: shrink(negative) or expand(positive) the bounding box
:param img: original image
:param corners: ((column_min, row_min),(column_max, row_max))
:return: list of clipping images
"""
clips = []
for component in components:
clip = component.compo_clipping(img, pad=pad)
clips.append(clip)
if show:
cv2.imshow('clipping', clip)
cv2.waitKey()
return clips
def dissemble_clip_img_hollow(clip_root, org, compos):
if os.path.exists(clip_root):
shutil.rmtree(clip_root)
os.mkdir(clip_root)
cls_dirs = []
bkg = org.copy()
hollow_out = np.ones(bkg.shape[:2], dtype=np.uint8) * 255
for compo in compos:
cls = compo.category
c_root = pjoin(clip_root, cls)
c_path = pjoin(c_root, str(compo.id) + '.jpg')
if cls not in cls_dirs:
os.mkdir(c_root)
cls_dirs.append(cls)
clip = compo.compo_clipping(org)
cv2.imwrite(c_path, clip)
col_min, row_min, col_max, row_max = compo.put_bbox()
hollow_out[row_min: row_max, col_min: col_max] = 0
bkg = cv2.merge((bkg, hollow_out))
cv2.imwrite(os.path.join(clip_root, 'bkg.png'), bkg)
def dissemble_clip_img_fill(clip_root, org, compos, flag='most'):
def average_pix_around(pad=6, offset=3):
up = row_min - pad if row_min - pad >= 0 else 0
left = col_min - pad if col_min - pad >= 0 else 0
bottom = row_max + pad if row_max + pad < org.shape[0] - 1 else org.shape[0] - 1
right = col_max + pad if col_max + pad < org.shape[1] - 1 else org.shape[1] - 1
average = []
for i in range(3):
avg_up = np.average(org[up:row_min - offset, left:right, i])
avg_bot = np.average(org[row_max + offset:bottom, left:right, i])
avg_left = np.average(org[up:bottom, left:col_min - offset, i])
avg_right = np.average(org[up:bottom, col_max + offset:right, i])
average.append(int((avg_up + avg_bot + avg_left + avg_right)/4))
return average
def most_pix_around(pad=6, offset=2):
up = row_min - pad if row_min - pad >= 0 else 0
left = col_min - pad if col_min - pad >= 0 else 0
bottom = row_max + pad if row_max + pad < org.shape[0] - 1 else org.shape[0] - 1
right = col_max + pad if col_max + pad < org.shape[1] - 1 else org.shape[1] - 1
most = []
for i in range(3):
val = np.concatenate((org[up:row_min - offset, left:right, i].flatten(),
org[row_max + offset:bottom, left:right, i].flatten(),
org[up:bottom, left:col_min - offset, i].flatten(),
org[up:bottom, col_max + offset:right, i].flatten()))
# print(val)
# print(np.argmax(np.bincount(val)))
most.append(int(np.argmax(np.bincount(val))))
return most
if os.path.exists(clip_root):
shutil.rmtree(clip_root)
os.mkdir(clip_root)
cls_dirs = []
bkg = org.copy()
for compo in compos:
cls = compo.category
c_root = pjoin(clip_root, cls)
c_path = pjoin(c_root, str(compo.id) + '.jpg')
if cls not in cls_dirs:
os.mkdir(c_root)
cls_dirs.append(cls)
clip = compo.compo_clipping(org)
cv2.imwrite(c_path, clip)
col_min, row_min, col_max, row_max = compo.put_bbox()
if flag == 'average':
color = average_pix_around()
elif flag == 'most':
color = most_pix_around()
cv2.rectangle(bkg, (col_min, row_min), (col_max, row_max), color, -1)
cv2.imwrite(os.path.join(clip_root, 'bkg.png'), bkg)

113
UIED/detect_compo/deprecated/ocr_classify_text.py Normal file
View File

@@ -0,0 +1,113 @@
import pytesseract as pyt
import cv2
import lib_ip.ip_draw as draw
from config.CONFIG_UIED import Config
C = Config()
def is_text(img, min_word_area, show=False):
broad = img.copy()
area_word = 0
area_total = img.shape[0] * img.shape[1]
try:
# ocr text detection
data = pyt.image_to_data(img).split('\n')
except:
print(img.shape)
return -1
word = []
for d in data[1:]:
d = d.split()
if d[-1] != '-1':
if d[-1] != '-' and d[-1] != '—' and int(d[-3]) < 50 and int(d[-4]) < 100:
word.append(d)
t_l = (int(d[-6]), int(d[-5]))
b_r = (int(d[-6]) + int(d[-4]), int(d[-5]) + int(d[-3]))
area_word += int(d[-4]) * int(d[-3])
cv2.rectangle(broad, t_l, b_r, (0,0,255), 1)
if show:
for d in word: print(d)
print(area_word/area_total)
cv2.imshow('a', broad)
cv2.waitKey(0)
cv2.destroyAllWindows()
# no text in this clip or relatively small text area
if len(word) == 0 or area_word/area_total < min_word_area:
return False
return True
def text_detection(org, img_clean):
try:
data = pyt.image_to_data(img_clean).split('\n')
except:
return org, None
corners_word = []
for d in data[1:]:
d = d.split()
if d[-1] != '-1':
if d[-1] != '-' and d[-1] != '—' and 5 < int(d[-3]) < 40 and 5 < int(d[-4]) < 100:
t_l = (int(d[-6]), int(d[-5]))
b_r = (int(d[-6]) + int(d[-4]), int(d[-5]) + int(d[-3]))
corners_word.append((t_l, b_r))
return corners_word
# def text_merge_word_into_line(org, corners_word, max_words_gap=C.THRESHOLD_TEXT_MAX_WORD_GAP):
#
# def is_in_line(word):
# for i in range(len(lines)):
# line = lines[i]
# # at the same row
# if abs(line['center'][1] - word['center'][1]) < max_words_gap:
# # small gap between words
# if (abs(line['center'][0] - word['center'][0]) - abs(line['width']/2 + word['width']/2)) < max_words_gap:
# return i
# return -1
#
# def merge_line(word, index):
# line = lines[index]
# # on the left
# if word['center'][0] < line['center'][0]:
# line['col_min'] = word['col_min']
# # on the right
# else:
# line['col_max'] = word['col_max']
# line['row_min'] = min(line['row_min'], word['row_min'])
# line['row_max'] = max(line['row_max'], word['row_max'])
# line['width'] = line['col_max'] - line['col_min']
# line['height'] = line['row_max'] - line['row_min']
# line['center'] = ((line['col_max'] + line['col_min'])/2, (line['row_max'] + line['row_min'])/2)
#
# words = []
# for corner in corners_word:
# word = {}
# (top_left, bottom_right) = corner
# (col_min, row_min) = top_left
# (col_max, row_max) = bottom_right
# word['col_min'], word['col_max'], word['row_min'], word['row_max'] = col_min, col_max, row_min, row_max
# word['height'] = row_max - row_min
# word['width'] = col_max - col_min
# word['center'] = ((col_max + col_min)/2, (row_max + row_min)/2)
# words.append(word)
#
# lines = []
# for word in words:
# line_index = is_in_line(word)
# # word is in current line
# if line_index != -1:
# merge_line(word, line_index)
# # word is not in current line
# else:
# # this single word as a new line
# lines.append(word)
#
# corners_line = []
# for l in lines:
# corners_line.append(((l['col_min'], l['row_min']), (l['col_max'], l['row_max'])))
# return corners_line

127
UIED/detect_compo/ip_region_proposal.py Normal file
View File

@@ -0,0 +1,127 @@
import cv2
from os.path import join as pjoin
import time
import json
import numpy as np
import detect_compo.lib_ip.ip_preprocessing as pre
import detect_compo.lib_ip.ip_draw as draw
import detect_compo.lib_ip.ip_detection as det
import detect_compo.lib_ip.file_utils as file
import detect_compo.lib_ip.Component as Compo
from config.CONFIG_UIED import Config
C = Config()
def resolve_uicompo_containment(uicompos):
"""
Resolves containment issues among UI components.
If a component's bounding box is fully contained within another's, it is removed.
"""
def contains(bbox_a, bbox_b):
"""Checks if bbox_a completely contains bbox_b."""
return bbox_a.col_min <= bbox_b.col_min and \
bbox_a.row_min <= bbox_b.row_min and \
bbox_a.col_max >= bbox_b.col_max and \
bbox_a.row_max >= bbox_b.row_max
compos_to_remove = set()
for i, compo1 in enumerate(uicompos):
for j, compo2 in enumerate(uicompos):
if i == j:
continue
# Check if compo1 contains compo2
if contains(compo1.bbox, compo2.bbox):
compos_to_remove.add(j)
# Filter out the contained components
final_compos = [compo for i, compo in enumerate(uicompos) if i not in compos_to_remove]
if len(final_compos) < len(uicompos):
print(f"Containment resolved: Removed {len(uicompos) - len(final_compos)} contained components.")
return final_compos
def nesting_inspection(org, grey, compos, ffl_block):
'''
Inspect all big compos through block division by flood-fill
:param ffl_block: gradient threshold for flood-fill
:return: nesting compos
'''
nesting_compos = []
for i, compo in enumerate(compos):
if compo.height > 50:
replace = False
clip_grey = compo.compo_clipping(grey)
n_compos = det.nested_components_detection(clip_grey, org, grad_thresh=ffl_block, show=False)
Compo.cvt_compos_relative_pos(n_compos, compo.bbox.col_min, compo.bbox.row_min)
for n_compo in n_compos:
if n_compo.redundant:
compos[i] = n_compo
replace = True
break
if not replace:
nesting_compos += n_compos
return nesting_compos
def compo_detection(input_img_path, output_root, uied_params,
resize_by_height=800, classifier=None, show=False, wai_key=0):
start = time.perf_counter()
name = input_img_path.split('/')[-1][:-4] if '/' in input_img_path else input_img_path.split('\\')[-1][:-4]
ip_root = file.build_directory(pjoin(output_root, "ip"))
# *** Step 1 *** pre-processing: read img -> get binary map
org, grey = pre.read_img(input_img_path, resize_by_height)
binary = pre.binarization(org, grad_min=int(uied_params['min-grad']))
# *** Step 2 *** element detection
det.rm_line(binary, show=show, wait_key=wai_key)
uicompos = det.component_detection(binary, min_obj_area=int(uied_params['min-ele-area']))
# *** Step 3 *** results refinement
uicompos = det.compo_filter(uicompos, min_area=int(uied_params['min-ele-area']), img_shape=binary.shape)
uicompos = det.merge_intersected_compos(uicompos)
det.compo_block_recognition(binary, uicompos)
if uied_params['merge-contained-ele']:
uicompos = det.rm_contained_compos_not_in_block(uicompos)
Compo.compos_update(uicompos, org.shape)
Compo.compos_containment(uicompos)
# *** Step 4 ** nesting inspection: check if big compos have nesting element
uicompos += nesting_inspection(org, grey, uicompos, ffl_block=uied_params['ffl-block'])
Compo.compos_update(uicompos, org.shape)
draw.draw_bounding_box(org, uicompos, show=show, name='merged compo', write_path=pjoin(ip_root, name + '.jpg'), wait_key=wai_key)
# *** Step 5 *** image inspection: recognize image -> remove noise in image -> binarize with larger threshold and reverse -> rectangular compo detection
# if classifier is not None:
# classifier['Image'].predict(seg.clipping(org, uicompos), uicompos)
# draw.draw_bounding_box_class(org, uicompos, show=show)
# uicompos = det.rm_noise_in_large_img(uicompos, org)
# draw.draw_bounding_box_class(org, uicompos, show=show)
# det.detect_compos_in_img(uicompos, binary_org, org)
# draw.draw_bounding_box(org, uicompos, show=show)
# if classifier is not None:
# classifier['Noise'].predict(seg.clipping(org, uicompos), uicompos)
# draw.draw_bounding_box_class(org, uicompos, show=show)
# uicompos = det.rm_noise_compos(uicompos)
# *** Step 6 *** element classification: all category classification
# if classifier is not None:
# classifier['Elements'].predict([compo.compo_clipping(org) for compo in uicompos], uicompos)
# draw.draw_bounding_box_class(org, uicompos, show=show, name='cls', write_path=pjoin(ip_root, 'result.jpg'))
# draw.draw_bounding_box_class(org, uicompos, write_path=pjoin(output_root, 'result.jpg'))
# *** Step 7 *** save detection result
Compo.compos_update(uicompos, org.shape)
# *** Step 8 *** resolve containment issues among UI components
uicompos = resolve_uicompo_containment(uicompos)
file.save_corners_json(pjoin(ip_root, name + '.json'), uicompos)
print("[Compo Detection Completed in %.3f s] Input: %s Output: %s" % (time.perf_counter() - start, input_img_path, pjoin(ip_root, name + '.json')))
return uicompos

122
UIED/detect_compo/lib_ip/Bbox.py Normal file
View File

@@ -0,0 +1,122 @@
import numpy as np
import detect_compo.lib_ip.ip_draw as draw
class Bbox:
def __init__(self, col_min, row_min, col_max, row_max):
self.col_min = col_min
self.row_min = row_min
self.col_max = col_max
self.row_max = row_max
self.width = col_max - col_min
self.height = row_max - row_min
self.box_area = self.width * self.height
def put_bbox(self):
return self.col_min, self.row_min, self.col_max, self.row_max
def bbox_cal_area(self):
self.box_area = self.width * self.height
return self.box_area
def bbox_relation(self, bbox_b):
"""
:return: -1 : a in b
0 : a, b are not intersected
1 : b in a
2 : a, b are identical or intersected
"""
col_min_a, row_min_a, col_max_a, row_max_a = self.put_bbox()
col_min_b, row_min_b, col_max_b, row_max_b = bbox_b.put_bbox()
# if a is in b
if col_min_a > col_min_b and row_min_a > row_min_b and col_max_a < col_max_b and row_max_a < row_max_b:
return -1
# if b is in a
elif col_min_a < col_min_b and row_min_a < row_min_b and col_max_a > col_max_b and row_max_a > row_max_b:
return 1
# a and b are non-intersect
elif (col_min_a > col_max_b or row_min_a > row_max_b) or (col_min_b > col_max_a or row_min_b > row_max_a):
return 0
# intersection
else:
return 2
def bbox_relation_nms(self, bbox_b, bias=(0, 0)):
'''
Calculate the relation between two rectangles by nms
:return: -1 : a in b
0 : a, b are not intersected
1 : b in a
2 : a, b are intersected
'''
col_min_a, row_min_a, col_max_a, row_max_a = self.put_bbox()
col_min_b, row_min_b, col_max_b, row_max_b = bbox_b.put_bbox()
bias_col, bias_row = bias
# get the intersected area
col_min_s = max(col_min_a - bias_col, col_min_b - bias_col)
row_min_s = max(row_min_a - bias_row, row_min_b - bias_row)
col_max_s = min(col_max_a + bias_col, col_max_b + bias_col)
row_max_s = min(row_max_a + bias_row, row_max_b + bias_row)
w = np.maximum(0, col_max_s - col_min_s)
h = np.maximum(0, row_max_s - row_min_s)
inter = w * h
area_a = (col_max_a - col_min_a) * (row_max_a - row_min_a)
area_b = (col_max_b - col_min_b) * (row_max_b - row_min_b)
iou = inter / (area_a + area_b - inter)
ioa = inter / self.box_area
iob = inter / bbox_b.box_area
if iou == 0 and ioa == 0 and iob == 0:
return 0
# import lib_ip.ip_preprocessing as pre
# org_iou, _ = pre.read_img('uied/data/input/7.jpg', 800)
# print(iou, ioa, iob)
# board = draw.draw_bounding_box(org_iou, [self], color=(255,0,0))
# draw.draw_bounding_box(board, [bbox_b], color=(0,255,0), show=True)
# contained by b
if ioa >= 1:
return -1
# contains b
if iob >= 1:
return 1
# not intersected with each other
# intersected
if iou >= 0.02 or iob > 0.2 or ioa > 0.2:
return 2
# if iou == 0:
# print('ioa:%.5f; iob:%.5f; iou:%.5f' % (ioa, iob, iou))
return 0
def bbox_cvt_relative_position(self, col_min_base, row_min_base):
'''
Convert to relative position based on base coordinator
'''
self.col_min += col_min_base
self.col_max += col_min_base
self.row_min += row_min_base
self.row_max += row_min_base
def bbox_merge(self, bbox_b):
'''
Merge two intersected bboxes
'''
col_min_a, row_min_a, col_max_a, row_max_a = self.put_bbox()
col_min_b, row_min_b, col_max_b, row_max_b = bbox_b.put_bbox()
col_min = min(col_min_a, col_min_b)
col_max = max(col_max_a, col_max_b)
row_min = min(row_min_a, row_min_b)
row_max = max(row_max_a, row_max_b)
new_bbox = Bbox(col_min, row_min, col_max, row_max)
return new_bbox
def bbox_padding(self, image_shape, pad):
row, col = image_shape[:2]
self.col_min = max(self.col_min - pad, 0)
self.col_max = min(self.col_max + pad, col)
self.row_min = max(self.row_min - pad, 0)
self.row_max = min(self.row_max + pad, row)

238
UIED/detect_compo/lib_ip/Component.py Normal file
View File

@@ -0,0 +1,238 @@
from detect_compo.lib_ip.Bbox import Bbox
import detect_compo.lib_ip.ip_draw as draw
import cv2
def cvt_compos_relative_pos(compos, col_min_base, row_min_base):
for compo in compos:
compo.compo_relative_position(col_min_base, row_min_base)
def compos_containment(compos):
for i in range(len(compos) - 1):
for j in range(i + 1, len(compos)):
relation = compos[i].compo_relation(compos[j])
if relation == -1:
compos[j].contain.append(i)
if relation == 1:
compos[i].contain.append(j)
def compos_update(compos, org_shape):
for i, compo in enumerate(compos):
# start from 1, id 0 is background
compo.compo_update(i + 1, org_shape)
class Component:
def __init__(self, region, image_shape):
self.id = None
self.region = region
self.boundary = self.compo_get_boundary()
self.bbox = self.compo_get_bbox()
self.bbox_area = self.bbox.box_area
self.region_area = len(region)
self.width = len(self.boundary[0])
self.height = len(self.boundary[2])
self.image_shape = image_shape
self.area = self.width * self.height
self.category = 'Compo'
self.contain = []
self.rect_ = None
self.line_ = None
self.redundant = False
def compo_update(self, id, org_shape):
self.id = id
self.image_shape = org_shape
self.width = self.bbox.width
self.height = self.bbox.height
self.bbox_area = self.bbox.box_area
self.area = self.width * self.height
def put_bbox(self):
return self.bbox.put_bbox()
def compo_update_bbox_area(self):
self.bbox_area = self.bbox.bbox_cal_area()
def compo_get_boundary(self):
'''
get the bounding boundary of an object(region)
boundary: [top, bottom, left, right]
-> up, bottom: (column_index, min/max row border)
-> left, right: (row_index, min/max column border) detect range of each row
'''
border_up, border_bottom, border_left, border_right = {}, {}, {}, {}
for point in self.region:
# point: (row_index, column_index)
# up, bottom: (column_index, min/max row border) detect range of each column
if point[1] not in border_up or border_up[point[1]] > point[0]:
border_up[point[1]] = point[0]
if point[1] not in border_bottom or border_bottom[point[1]] < point[0]:
border_bottom[point[1]] = point[0]
# left, right: (row_index, min/max column border) detect range of each row
if point[0] not in border_left or border_left[point[0]] > point[1]:
border_left[point[0]] = point[1]
if point[0] not in border_right or border_right[point[0]] < point[1]:
border_right[point[0]] = point[1]
boundary = [border_up, border_bottom, border_left, border_right]
# descending sort
for i in range(len(boundary)):
boundary[i] = [[k, boundary[i][k]] for k in boundary[i].keys()]
boundary[i] = sorted(boundary[i], key=lambda x: x[0])
return boundary
def compo_get_bbox(self):
"""
Get the top left and bottom right points of boundary
:param boundaries: boundary: [top, bottom, left, right]
-> up, bottom: (column_index, min/max row border)
-> left, right: (row_index, min/max column border) detect range of each row
:return: corners: [(top_left, bottom_right)]
-> top_left: (column_min, row_min)
-> bottom_right: (column_max, row_max)
"""
col_min, row_min = (int(min(self.boundary[0][0][0], self.boundary[1][-1][0])), int(min(self.boundary[2][0][0], self.boundary[3][-1][0])))
col_max, row_max = (int(max(self.boundary[0][0][0], self.boundary[1][-1][0])), int(max(self.boundary[2][0][0], self.boundary[3][-1][0])))
bbox = Bbox(col_min, row_min, col_max, row_max)
return bbox
def compo_is_rectangle(self, min_rec_evenness, max_dent_ratio, test=False):
'''
detect if an object is rectangle by evenness and dent of each border
'''
dent_direction = [1, -1, 1, -1] # direction for convex
flat = 0
parameter = 0
for n, border in enumerate(self.boundary):
parameter += len(border)
# dent detection
pit = 0 # length of pit
depth = 0 # the degree of surface changing
if n <= 1:
adj_side = max(len(self.boundary[2]), len(self.boundary[3])) # get maximum length of adjacent side
else:
adj_side = max(len(self.boundary[0]), len(self.boundary[1]))
# -> up, bottom: (column_index, min/max row border)
# -> left, right: (row_index, min/max column border) detect range of each row
abnm = 0
for i in range(int(3 + len(border) * 0.02), len(border) - 1):
# calculate gradient
difference = border[i][1] - border[i + 1][1]
# the degree of surface changing
depth += difference
# ignore noise at the start of each direction
if i / len(border) < 0.08 and (dent_direction[n] * difference) / adj_side > 0.5:
depth = 0 # reset
# print(border[i][1], i / len(border), depth, (dent_direction[n] * difference) / adj_side)
# if the change of the surface is too large, count it as part of abnormal change
if abs(depth) / adj_side > 0.3:
abnm += 1 # count the size of the abnm
# if the abnm is too big, the shape should not be a rectangle
if abnm / len(border) > 0.1:
if test:
print('abnms', abnm, abnm / len(border))
draw.draw_boundary([self], self.image_shape, show=True)
self.rect_ = False
return False
continue
else:
# reset the abnm if the depth back to normal
abnm = 0
# if sunken and the surface changing is large, then counted as pit
if dent_direction[n] * depth < 0 and abs(depth) / adj_side > 0.15:
pit += 1
continue
# if the surface is not changing to a pit and the gradient is zero, then count it as flat
if abs(depth) < 1 + adj_side * 0.015:
flat += 1
if test:
print(depth, adj_side, flat)
# if the pit is too big, the shape should not be a rectangle
if pit / len(border) > max_dent_ratio:
if test:
print('pit', pit, pit / len(border))
draw.draw_boundary([self], self.image_shape, show=True)
self.rect_ = False
return False
if test:
print(flat / parameter, '\n')
draw.draw_boundary([self], self.image_shape, show=True)
# ignore text and irregular shape
if self.height / self.image_shape[0] > 0.3:
min_rec_evenness = 0.85
if (flat / parameter) < min_rec_evenness:
self.rect_ = False
return False
self.rect_ = True
return True
def compo_is_line(self, min_line_thickness):
"""
Check this object is line by checking its boundary
:param boundary: boundary: [border_top, border_bottom, border_left, border_right]
-> top, bottom: list of (column_index, min/max row border)
-> left, right: list of (row_index, min/max column border) detect range of each row
:param min_line_thickness:
:return: Boolean
"""
# horizontally
slim = 0
for i in range(self.width):
if abs(self.boundary[1][i][1] - self.boundary[0][i][1]) <= min_line_thickness:
slim += 1
if slim / len(self.boundary[0]) > 0.93:
self.line_ = True
return True
# vertically
slim = 0
for i in range(self.height):
if abs(self.boundary[2][i][1] - self.boundary[3][i][1]) <= min_line_thickness:
slim += 1
if slim / len(self.boundary[2]) > 0.93:
self.line_ = True
return True
self.line_ = False
return False
def compo_relation(self, compo_b, bias=(0, 0)):
"""
:return: -1 : a in b
0 : a, b are not intersected
1 : b in a
2 : a, b are identical or intersected
"""
return self.bbox.bbox_relation_nms(compo_b.bbox, bias)
def compo_relative_position(self, col_min_base, row_min_base):
'''
Convert to relative position based on base coordinator
'''
self.bbox.bbox_cvt_relative_position(col_min_base, row_min_base)
def compo_merge(self, compo_b):
self.bbox = self.bbox.bbox_merge(compo_b.bbox)
self.compo_update(self.id, self.image_shape)
def compo_clipping(self, img, pad=0, show=False):
(column_min, row_min, column_max, row_max) = self.put_bbox()
column_min = max(column_min - pad, 0)
column_max = min(column_max + pad, img.shape[1])
row_min = max(row_min - pad, 0)
row_max = min(row_max + pad, img.shape[0])
clip = img[row_min:row_max, column_min:column_max]
if show:
cv2.imshow('clipping', clip)
cv2.waitKey()
return clip

BIN
UIED/detect_compo/lib_ip/__pycache__/Bbox.cpython-312.pyc Normal file
View File

Binary file not shown.

BIN
UIED/detect_compo/lib_ip/__pycache__/Bbox.cpython-35.pyc Normal file
View File

Binary file not shown.

BIN
UIED/detect_compo/lib_ip/__pycache__/Block.cpython-35.pyc Normal file
View File

Binary file not shown.

BIN
UIED/detect_compo/lib_ip/__pycache__/Component.cpython-312.pyc Normal file
View File

Binary file not shown.

BIN
UIED/detect_compo/lib_ip/__pycache__/Component.cpython-35.pyc Normal file
View File

Binary file not shown.

BIN
UIED/detect_compo/lib_ip/__pycache__/file_utils.cpython-312.pyc Normal file
View File

Binary file not shown.

BIN
UIED/detect_compo/lib_ip/__pycache__/file_utils.cpython-35.pyc Normal file
View File

Binary file not shown.

BIN
UIED/detect_compo/lib_ip/__pycache__/ip_detection.cpython-312.pyc Normal file
View File

Binary file not shown.

BIN
UIED/detect_compo/lib_ip/__pycache__/ip_detection.cpython-35.pyc Normal file
View File

Binary file not shown.

BIN
UIED/detect_compo/lib_ip/__pycache__/ip_draw.cpython-312.pyc Normal file
View File

Binary file not shown.

BIN
UIED/detect_compo/lib_ip/__pycache__/ip_draw.cpython-35.pyc Normal file
View File

Binary file not shown.

BIN
UIED/detect_compo/lib_ip/__pycache__/ip_preprocessing.cpython-312.pyc Normal file
View File

Binary file not shown.

Some files were not shown because too many files have changed in this diff Show More