AppleScript C Perl Shell Xcode Other

Automatic image text extraction using Live Text in Preview on Mac

Post Reply
coding / applescript     Views: 273Prev .. Next
Automatic image text extraction using Live Text in Preview on MacPosted: Thursday, July 20, 2023 [21:25:04] - 1
rootPosted by:rootMember Since:
June 16 2010
Posts: 357
Reasons why automation needed to extract text from images may vary.
Preview's Live Text works very good and our project required to read a lot of receipts from time-to-time but we do not want to do it manually.
Automatic image text extraction using Live Text in Preview on Mac
The main obstacle in making it work was necessity to select the portion of the image with text or plain clicking on the image to make Live Text work.
Old trusty Python worked great using Quartz module but module requires installation and searching for the proper installation suggested lengthy process.

A "cliclick" program installation appears to be straight-forward with brew:
View Codebrew install cliclick


Should it work? Unfortunately not all the time unless AppleScript generates two clicks.
Here is the code we came-up with to get the image text by AppleScript automatically every time:
View Code
-- UNIX path is easier to write and will always be correct, no matter how you called your Hard Drive
my $src = 'set p to "/path/to/image/file/image.jpg"
-- convert UNIX path to AppleScript readable folder and file
set f to POSIX file p as alias

-- open image file
tell application "Preview"
activate
open f
end tell
tell application "System Events"
tell its process "Preview"
-- Select appropriate tools in Preview
click menu item "Text Selection" of menu "Tools" of menu bar 1
delay 1
-- and a Markup Toolbar
keystroke "a" using {command down, shift down}
delay 1
end tell
end tell
delay 2

tell application "Preview"
-- get the size and position of the Preview window
set bnd to bounds of the window 1
set lp to item 1 of bnd
set tp to item 2 of bnd
set width to item 3 of bnd
set height to item 4 of bnd
end tell
-- get the position in the center of the image to click
set wp to width / 2 -- get a half width of Preview window
set wp to wp + lp -- add to it a left margine of the screen
set hp to height / 2 -- the same for height
set hp to hp + tp
delay 1
-- remove the decimal point from coordinates, cliclick do not like it and throws the error
set wp to do shell script "echo " & wp & "|/path/to/perl -n -e 'if($_ =~ m/\\./){s/(\\d+)\\.(\\d{1,2})/$1/;}print;'"
set hp to do shell script "echo " & hp & "|/path/to/perl -n -e 'if($_ =~ m/\\./){s/(\\d+)\\.(\\d{1,2})/$1/;}print;'"
-- click in the center of the image
do shell script "/path/to/cliclick c:" & wp & "," & hp
-- change the coordinates slightly and click again
set wp to wp - 5
set hp to hp - 5
delay 1
do shell script "/path/to/cliclick c:" & wp & "," & hp
delay 2
-- now select all the text and copy it to a clipboard
tell application "System Events"
tell application process "Preview"
keystroke "a" using command down
delay 1
keystroke "c" using {command down}
delay 2
-- exit Preview
keystroke "q" using {command down}
delay 1
end tell
end tell
-- copy text to a text file for further processing
do shell script "/path/to/pbpaste > /path/to/text/file/text.txt"


The code runs from Perl script and a cron job runs the Perl script.
Please do not forget to substitute "/path/to" accordingly.There's no place like ~
coding / applescriptPrev .. Next
 
Post Reply
Home - Coding: AppleScript C Perl Shell Xcode Other
Our Telegram Group