Automatic image text extraction using Live Text in Preview on Mac
Post ReplyAutomatic image text extraction using Live Text in Preview on MacPosted: Thursday, July 20, 2023 [21:25:04] - 1
Reasons why automation needed to extract text from images may vary. Preview's Live Text works very good and our project required to read a lot of receipts from time-to-time but we do not want to do it manually. The main obstacle in making it work was necessity to select the portion of the image with text or plain clicking on the image to make Live Text work. Old trusty Python worked great using Quartz module but module requires installation and searching for the proper installation suggested lengthy process. A "cliclick" program installation appears to be straight-forward with brew: Should it work? Unfortunately not all the time unless AppleScript generates two clicks. Here is the code we came-up with to get the image text by AppleScript automatically every time: -- UNIX path is easier to write and will always be correct, no matter how you called your Hard Drive my $src = 'set p to "/path/to/image/file/image.jpg" -- convert UNIX path to AppleScript readable folder and file set f to POSIX file p as alias -- open image file tell application "Preview" activate open f end tell tell application "System Events" tell its process "Preview" -- Select appropriate tools in Preview click menu item "Text Selection" of menu "Tools" of menu bar 1 delay 1 -- and a Markup Toolbar keystroke "a" using {command down, shift down} delay 1 end tell end tell delay 2 tell application "Preview" -- get the size and position of the Preview window set bnd to bounds of the window 1 set lp to item 1 of bnd set tp to item 2 of bnd set width to item 3 of bnd set height to item 4 of bnd end tell -- get the position in the center of the image to click set wp to width / 2 -- get a half width of Preview window set wp to wp + lp -- add to it a left margine of the screen set hp to height / 2 -- the same for height set hp to hp + tp delay 1 -- remove the decimal point from coordinates, cliclick do not like it and throws the error set wp to do shell script "echo " & wp & "|/path/to/perl -n -e 'if($_ =~ m/\\./){s/(\\d+)\\.(\\d{1,2})/$1/;}print;'" set hp to do shell script "echo " & hp & "|/path/to/perl -n -e 'if($_ =~ m/\\./){s/(\\d+)\\.(\\d{1,2})/$1/;}print;'" -- click in the center of the image do shell script "/path/to/cliclick c:" & wp & "," & hp -- change the coordinates slightly and click again set wp to wp - 5 set hp to hp - 5 delay 1 do shell script "/path/to/cliclick c:" & wp & "," & hp delay 2 -- now select all the text and copy it to a clipboard tell application "System Events" tell application process "Preview" keystroke "a" using command down delay 1 keystroke "c" using {command down} delay 2 -- exit Preview keystroke "q" using {command down} delay 1 end tell end tell -- copy text to a text file for further processing do shell script "/path/to/pbpaste > /path/to/text/file/text.txt" The code runs from Perl script and a cron job runs the Perl script. Please do not forget to substitute "/path/to" accordingly. |