A Brief Tour of a Riven OCR app I wrote (and some other kinda neat stuff)

[sly_squash]

Watch a (Slick) Quick Tools Demonstration on Youtube!

 

Watch Riven OCR In-Depth Walkthrough on Youtube!

 

Abstract:

So I wrote a riven OCR app to help alleviate some of the pain of posting rivens online. For those who don't know, OCR stands for "Optical Character Recognition". It's a technology used to extract text from images and is commonly used to transform physical books and other publications into digital ones with fully-searchable text.

Here it is, in all it's glory.

 

How do you use it?

There are two ways to get rivens into the application:

1. Screenshots
   • Make sure the Riven OCR app is running. It registers the global hotkey CTRL+SHIFT+B on application startup.
   • Open up a riven link in Warframe, such as [Komorex Acridra] like I did here: https://i.imgur.com/N9AOVgg.png
   • Press the hotkey.
   • Riven OCR will take a screenshot, extract out the the bit in the middle containing the riven, and save this image as a temporary file.
   • Riven OCR will automatically scan the riven, extract as much information as it can, and display the results on the right.
   • If the Rename Screenshots by Riven Name option is checked, it will automatically rename the temporary file in accordance with the riven name, as is shown in the following screenshot: https://i.imgur.com/03H3dmn.png
2. Previously-saved riven image files
   • Press the giant Load button.
   • Select the riven image file from the directory of rivens that were previously entered via screenshot.
   • Riven OCR will verify that the image is in the correct expected dimensions and, if it is, extract the riven information and display it on the screen.

Cool, but what now?

Scanning riven files and saving them to your directory is great and all, but real power of Riven OCR is in helping to quickly and effortlessly post rivens. 

1. Verify that Riven OCR has properly scanned all the properties of the riven.
2. Hit the Post to Riven.Market (or Post to Warframe.Market button if you're so inclined)
3. If Riven OCR hasn't already helped post a riven, it'll open up it's own copy of Google Chrome and navigate to the corresponding page. If it has, it'll be even faster and just keep using it's already-opened browser instance.
4. Riven OCR now automatically fills out all the fields of the page for you. 
5. Verify that the riven it has entered into the market page is the correct per the riven you loaded.
6. Hit "Next", enter the price, and hit "Post" to post the riven.

Here's a screenshot of what that looks like (you can see a clearer image at https://i.imgur.com/9MYQbTS.png)

 

 

But Riven.market already has a Discord bot to do Riven OCR. Why not use that?

I decided to make my own custom Riven OCR application instead of use the discord one for several reasons:

1. The Discord bot doesn't cover several fields that you still need to fill in manually. My application automates filling in all fields of the riven, including Polarity, Mod Rank, and Riven Name.
2. The Discord bot is cumbersome to use, as you have to take the screenshot yourself. upload the file, and copy-paste the import string. Riven OCR is two clicks: CTRL+SHIFT+B to take and import the screenshot and Post to Riven.Market to post the riven.
3. The Discord bot only works with Riven.Market. Riven OCR works with both Riven.Market and Warframe.Market and can be easily expanded to work with any other site (is wftrader still a thing?)

 

Don't you have to login with the Riven OCR browser every time?

As some of you probably expect, Riven OCR uses Selenium Webdriver (specifically ChromeDriver) to do the browser stuff. There's an option in there to bind the ChromeDriver instance against your default Chrome profile instead of using a blank profile every time. By doing so I never need to login to Riven.Market or Warframe.Market in Riven OCR's browser instance.

 

Will you get banned for this?

This is a very good question. The answer is that I'd probably be banned if I setup an app like this to run in batch mode (like auto-post 100 rivens back-to-back unattended over the course of like 2 minutes or something).

However, with this application I went to great lengths to make sure almost all the work that is done is client-side stuff. Riven OCR doesn't interact with Warframe in any way (it just takes a screenshot), nor does it do much of anything with Riven.Market or Warframe.Market other than load the page (and I've done a bit of Wiresharking to verify there doesn't look to be anything fishy about the Riven OCR ChromeDriver web requests). The bulk of what is does is completely offline (processing a riven image file and client-side form filling).

 

What's the deal with Black-and-White image?

The black-and-white image is another temporary intermediary file generated by Riven OCR to help it process the riven. OCR doesn't work super great on light-purple text on a black-ish background, so I essentially copy the riven image and alter the copy such that any pixel brighter than a certain threshold becomes white and darker than that threshold becomes black. However, in a riven's case, this means you end up with white text on a black background. Since OCR libraries are mostly optimized for scanning books, I invert the resulting image so that it display black text on a white background.

The black-and-white image is also how I extract information regarding Polarity and Mod Rank btw. I target very specific pixels in the resulting B&W bitmap and, depending on their color, can determine their values.

For example, with Mod Rank, every new rank essentially lights up a new tiny circle of pixels. These pixels become white in the B&W image, and are inverted to become black in the inverted image. I check if the pixels where the 8th rank is located are black first, and if they are then the Mod Rank is 8. If it's not, I check the pixels where the 7th rank is located, and so on.

 

You'll note that you don't see the Polarity in the top-right of the B&W image. This is because I extract that information before I clear out the "junk". Since the location of where the Riven Text actually starts varies depending on the riven, I have to set the location for the OCR libraries to run to start fairly high up vertically on the riven. However, since rivens have images and those images become very messy blotches of pixels that mess with OCR when converting to B&W, I have routines to clear up that mess so the OCR libraries don't have so much noisy input.

You'll note that I can leave the bottom of the riven pretty much as-is, because the line that contains Mastery Rank and Rolls is quite well-standardized and already clean so I can run a targets OCR scan on just that one "line" and be done with it.

By the way, you'll note there is an option to leave the Black and White temporary images (which can be useful during debugging) or delete them after use.

 

What's the deal with the "Raw Output" string in the bottom left?

Extracting the riven information is really only half the battle--the next half of the battle is parsing that output into something intelligible. The "Raw Output" is the string of text that describes the riven as per was lifted by the OCR (and my own specialized pixel-matching routines to grab Polarity and Mod Rank). 

It's not the final product. And while I do everything I can to give the OCR engine the cleanest possible input, the output I get still isn't perfect. It's messy, disorganized, and worst of all can contain noisy output (including distorted words, misinterpreted characters, etc.)

So, basically, if Riven OCR isn't displaying complete or correct information, I can look at the Raw Output string to see what it's working with and can usually spot the anomaly. One common problem are Elementals riven attributes, because OCR tries to interpret that little toxin graphic as a letter and reads it as a "3" or something. Another common "problem" is OCR detects the little circular arrow to the left of "Rolls" as a "O".

I call these minor annoyances "warts". After testing Riven OCR against hundreds of rivens, I have developed fairly robust guards against most common warts so everything still gets processed properly.

I also presume most fields won't be read accurately, and use Levenshtein distance algorithms against all the known possibilities to determine what should be the correct output. Levenshtein distance measures the number of characters that must be changed, added, or removed to transform a string of characters into another string. Thus, if OCR incorrectly reads a riven attribute as +102.6% Critical Damag, I run it against a list of all possible attributes and it will most closely match with Critical Damage. In that way I'm able to further clean up the output.

 

Why do you bother storing the Riven Images to disk?

I use them as input for another program. Check out my next post to see more.

 

Can I get this?

Maybe someday. It's currently heavily-tailored for my own computing environment; things like screen resolutions, quality settings, chrome profile locations would probably have to be generalized to work with other people's stuff. But who knows we'll see.

 

Do you have a girlfriend?

No. 😥

 

 

[sly_squash]

OK so here's are a few other apps I've written to help out with Warframe-y things.

 

CollageMaker

Purpose:  To generate priced Riven Collages that can be posted on Discord and various and get more eyes on your rivens.
Usage:

• Set the Image Directory to the location where the Riven Images are stored
• Set the Pricing Filename to be the location where the Riven Prices are stored (see Riven Tracker app)
• Choose the number of files and whether to maintain the temporary files created by the application
• Press "Make Collage"
• Check the image directory to find Collage.png.

Output:  See the image below (or check out the following image for the full version https://i.imgur.com/1a11zYH.jpg)

 

Riven Tracker

 

Purpose: 

1. To maintain a list of all rivens currently available for sale, including a simple text file that can be used by other applications (such as CollageMaker) to easily import riven and pricing information
2. To use the list of rivens to generate dynamic WTS messages that can be posted into trade chat.

Usage:

1. Open Riven Tracker. It will automatically register the CTRL+B global hotkey on startup
2. Enter in some rivens by entering the Riven Name and Price and hitting "Add"
3. Go to Warframe and hit CTRL+B
4. Riven Tracker will pick 5 random rivens and generate a well-formed WTS message. It also automatically copies the WTS information into the System Clipboard
5. Hit CTRL+V to paste the message that was generated and copied into clipboard into trade chat.

 

Store Tracker

Purpose: To easily track stores that have rotating, sellable stock so you can easily prioritize selling things that have been out-of-rotation the longest for maximum value.

Usage: 

1. Select "Baro" or "Nightwave" store (I also intend to add "Primes" in the future, though there is fairly good documentation for which of those have been out-of-rotation the longest available on the wiki)
2. Hit "Increase Wave" to indicate a new weekly rotation is available. This will increase the "Wave number" (i.e. number of rotations that each item has been unavailable) by one for every item
3. Type the name of the sellable mod, weapon, or whatever into the box and hit "Add"
4. If that item is already in the list, Store Tracker will update that item to be in "Wave 0" (i.e. currently available). If it's not, it will make a new entry.
5. Scroll to the top of the list to see what items have now been out-of-stock the longest. Maybe now is a good time to list your spares for maximal profit before they're back in stock again! 😁

 

RivenNameGenerator

Purpose: A simple command-line application I wrote to help improve Riven OCR output parsing that computes all possible Riven Names from the rules documented at https://warframe.fandom.com/wiki/Riven_Mods 

Usage: 

1. Craft an input file that contains the name of each possible riven attribute, its Prefix/Core and Suffix names, and whether it is a Gun-specific (Ex. Zoom), Melee-specific (Ex. Range), or general attribute (Ex. Critical Damage).
2. Type RivenNameGenerator <input filename> <output filename>
3. Open the output file to see all possible combinations
4. FYI according to my program there are 29,791 possible combinations. I've made them available in the following Pastebin. Feel free to write your own program and double-check my work!     https://pastebin.com/P9R2hphr

[Voltage]

This is an awesome project. It will be an awesome tool once completed. Keep it up!

One thing that would be really cool is if this could work like wf.info where it captures a Riven on your screen and grades it on an overlay based on the "Fits in" screen on the bottom right that shows a weapon (or weapon variant) to determine disposition and magnitude of attributes. Basically like the Discord bot without needing to screenshot and manually post it to a discord channel with a bot command.

[sly_squash]

Hmm, that's a neat idea. I could easily modify the input file to bring in "Disposition" with each Weapon and use that to evaluate Magnitude; however, that would require updating the input file every time there are disposition changes.

Alternatively, I could scrape https://warframe.fandom.com/wiki/Riven_Mods#Disposition to bring in the disposition values, but I wouldn't want to do that every startup because the wiki might not like that. I'd probably make a button "Update Dispositions" that does a scrape only when you press that button and updates it's own input file with the new dispos. 

From there I could calculate magnitudes to go along with the actual values so you know if you got a high roll. Maybe even color the box bright green for great magnitudes or red for weak ones.  Hmmmmmmmmmm 🤓

[Slayer-.]

Very awesome ideas, keep up the good work, will be good to see the final thing.

[sly_squash]

UPDATE: Added a youtube video demonstrating Riven OCR to the OP.  The video details basic operation and the latest updates (magnitudes + error correction).

[Slayer-.]

2 hours ago, sly_squash said:

UPDATE: Added a youtube video demonstrating Riven OCR to the OP.  The video details basic operation and the latest updates (magnitudes + error correction).

Just watched the video, that is just brilliant, well explained and laid out. 👍

[Voltage]

7 hours ago, sly_squash said:

UPDATE: Added a youtube video demonstrating Riven OCR to the OP.  The video details basic operation and the latest updates (magnitudes + error correction).

Very well done and very impressive. I can't wait to see further developments of this tool. 

[Psych0_Kratos]

"We will watch your career with great interest"... Cool tool

 

[bnuy]

Cool program but uh... Not git link or anything? I remembered someone showing me this and I was about to go through and relist my 100+ rivens then find out this program isn't actually available for download. Time to go spend hours listing all my rivens and wishing for death every moment because it's so freakin' painful to do...

Please release this program.

[sly_squash]

On 2021-07-05 at 4:45 AM, Maggy said:

Cool program but uh... Not git link or anything? I remembered someone showing me this and I was about to go through and relist my 100+ rivens then find out this program isn't actually available for download. Time to go spend hours listing all my rivens and wishing for death every moment because it's so freakin' painful to do...

Please release this program.

Very sorry that Riven OCR couldn't be there for you during the Grand Relisting. I fully understand the pain in having to list a stack of dozens of rivens at a time and really want to help.

The hold-up with Riven OCR wide distribution is in the licensing of Riven OCR's core libraries.

Sadly, it looks like licenses for the libraries I'm using right now would cost anywhere from $400 to several thousand dollars. However, because there's been significant interest in this project (I get a lot of PM's from people who have seen this thread and are interested in the software), I'm currently trying to transition away from IronOCR and into fully open-source OCR libraries like Tesseract. This will take some time, as Tesseract isn't as friendly as IronOCR and isn't as good at reading "dirty" input (i.e. stuff that isn't from a scanned book page). Also, to keep the accuracy high with these more sensitive libraries, I may need to try some other tricks, like to convert the riven into a grayscale image instead of black-and-white so that the character aliasing is better preserved.

Oh, and now warframe.market made some changes to their site to introduce Queens of Parvos weapons that breaks the warframe.market functionality of Riven OCR, so I gotta go back and fix that too. 😡

But that's par for the course. Thank you all for your interest, and please stay tuned! 

By the way, I've been developing another project on the side that I'll be posting about here very soon. It may be of some interest to all of you hardcore traders out there (and no OCR, so licensing shouldn't be as big an issue)...

[sly_squash]

Added a new video to OP that demo's most of the tools I've created so far, including the introduction of a new one I alluded to.. the Trade Request Notifier app!

I'll be doing an in-depth walkthrough of this tool very soon, but for now the new vid should give you the gist. Should help combat one of Warframe traders' biggest nemesis: the hopelessly afk seller on the other end.

[(PSN)GOTWIC]

Brilliant.

Now all you need is an ocr for tc

[sly_squash]

Made a full demonstration vid on the Trade Request Notifier, an app mentioned briefly in the compilation video I released previously.

 

 

 

[IDDQD21]

Did this ever get released?

 

Pretty amazing tool for someone who trades a lot of rivens.

[sly_squash]

10 hours ago, IDDQD21 said:

Did this ever get released?

 

Pretty amazing tool for someone who trades a lot of rivens.

It hasn't been released yet, but it is still under active development. I've had some hurdles come up recently (most notably my dad was diagnosed with cancer and my mom had a stroke), so a lot of my free time has gone to helping out with family stuff.

There's three key features that I'm currently in the process of adding and should be present by the final release:

1. Open Source OCR Engine: The initial implementation used IronOCR demo libraries as a proof-of-concept. They're quick and easy to use, but extremely expensive and unwieldy to license for wide distribution. I'm still working on migrating the engine to an open source engine. However, in my testing the open source alternatives are much less resilient to "dirty input", so I have to do much more to prep the image for OCR (Black-and-white conversion works fine for IronOCR, but the open source stuff seems to run better with a finely-tuned grayscale image, and I'm still sorting out how to produce those).
    
2. Warframe REST API: Right now, the program reads riven info from a plain-text data file I crafted that includes everything from dispositions to weapon families. This unfortunately does mean I have to update the data file manually whenever new weapons get added or dispositions get changed. I'm working on redirecting the data source to use Warframe's public programming API instead so that users can update their data files with the latest weapons and disposition info in the click of a button instead of needing me to update the data files and release a new version.
    
3. Sales Tracking: riven.market and warframe.market track how many rivens you've sold, but they don't track what those rivens looked like and how much they sold for. I'm working on adding this kind of tracking support to Riven OCR. The idea is that, before you sell a riven, you load its screenshot into Riven OCR then hit a new <Archive> button. Riven OCR will prompt you to input the price the riven sold at and generate/edit a .xls spreadsheet file, adding a new row at the top with the datetime, price, and riven info as a new line item. You can then go back and review the spreadsheet, leveraging excel to do data analysis (maybe you sort by Weapon Name so you can compare prices and properties of all the Phantasma rivens you've ever sold, for example.)

 

I've also been working on a more traditional Warframe video that is quite comprehensive and I think very useful. Stay tuned: I'll be releasing that video and reporting it here very soon.

[Slayer-.]

Keep up the great work. 👍

[sly_squash]

FYI, as I alluded to in my last post, I made a comprehensive riven unveiling guide. Full details are available in my forum post here:

 

 

[Eterud]

oh this looks neat, any updates/available to the public yet?

[sly_squash]

38 minutes ago, Eterud said:

oh this looks neat, any updates/available to the public yet?

It's getting much closer to being ready for public consumption. I've implemented #1 and #3 in my todo list.

#1 was to migrate to an open source OCR engine so I can distribute the application. I also made a number of optimizations so it takes around 0.5 seconds to scan a riven (as opposed to about a second per riven using the old proprietary engine).

#3 was sales tracking. Riven OCR now supports archiving sales orders to an excel file. Basically, you just hit a button, tell it the price it sold at, and it writes a new line item to the excel file with the date of purchase, sale price, and riven details.

Riven OCR now also supports all 16:9 aspect ratios (rather that JUST the exact resolution I use). Basically, it automatically resizes riven images to my resolution before doing the analysis and stuff.  I plan on adding support for 16:10 and 4:3 ratios in the near future.

Most recently I've been working on UX stuff. First, I want it to be easy to edit the riven properties in the panel so you can fix things that didn't scan properly. I've also been working on improving exception handling and such. Now it gives informative messages letting you know why the riven didn't post properly (such as that you weren't logged in) instead of just crashing. :P

[Eterud]

That all sounds great progress :) 

Can't wait to try it out. 

If you want to go "all the way" see if there's a reliable way to "guesstimate a riven's worth" 

[sly_squash]

3 hours ago, Eterud said:

If you want to go "all the way" see if there's a reliable way to "guesstimate a riven's worth" 

I would love to do that, but I would want a 3rd party to do the heavy lifting involved in appraising the riven.

In case you weren't around, we used to have a Discord server bot on Folren's server that would grade your riven based on weapon name and the specifics of your roll and spit out reasonably-accurate price ranges that the riven could sell for. It also gave a lot of good details about why it gave the riven the grade and prices it did. I could've likely setup Riven OCR to query that bot in the background to automatically populate the price for you, but sadly Folren's bot has been offline for years now after trying to accommodate significant changes DE made to (IIRC) the melee system.

If anyone knows of a similar service that has a public API Riven OCR could query for appraisals, I'd love to hear about it. Maybe semlar is enough but idk.

In the meantime, it may be worth knowing that I'm working on a youtube video where I discuss how I go about pricing rivens. According to riven.market, I've sold over 2.5k priced, rolled rivens in the past 2-3 years, so I feel like I have a good handle on a solid method for appraising your own stock.

[Eterud]

yey I'm back, for some reason couldn't get on the forums as after logging in it would tell me to log in again.


Anyways; After the folren's bot is gone, I've got no clue of other methods. only know of the semlar prices which is pretty much the prices that are listed in chat for the past 5 days.

 

[Eterud]

Just a friendly poke, as this tool still seems absolutely amazing for any trader.

(heck if you need beta testers or something I can help test, although I know nothing of coding) 

[krazox]

Hi, I'm also interested in the tools, mostly in the OCR autolisting.

Are there any news on the release date?
 

En 15/4/2022 a las 23:10, sly_squash dijo:

I would love to do that, but I would want a 3rd party to do the heavy lifting involved in appraising the riven.

In case you weren't around, we used to have a Discord server bot on Folren's server that would grade your riven based on weapon name and the specifics of your roll and spit out reasonably-accurate price ranges that the riven could sell for. It also gave a lot of good details about why it gave the riven the grade and prices it did. I could've likely setup Riven OCR to query that bot in the background to automatically populate the price for you, but sadly Folren's bot has been offline for years now after trying to accommodate significant changes DE made to (IIRC) the melee system.

If anyone knows of a similar service that has a public API Riven OCR could query for appraisals, I'd love to hear about it. Maybe semlar is enough but idk.

In the meantime, it may be worth knowing that I'm working on a youtube video where I discuss how I go about pricing rivens. According to riven.market, I've sold over 2.5k priced, rolled rivens in the past 2-3 years, so I feel like I have a good handle on a solid method for appraising your own stock.

Also as somewhat of a dev myself, I did a tool which had something similar to a guesstimator for a riven's price, instead of telling you the price, it tells you if the riven is worth its price based on its performance on the weapon giving you a rating of it, along with other wfm useful functions such as finding deals, a ducanator for full trades etc..

The approach I used is not perfect by any means, and something like Folren's bot is a thousand times better than what I could do but, in essence, I generated a json with profiles that depending on weapon types, uses (co-primer/heavy attacks/beams...) and specific cases (such as Rubico/Vectis) would give a punctuation based on stats and grades to said riven.

The stats are weighted depending on the weapon's stats and its mathematical value for dps or comfort to provide a more realistic result, and even though some weird results slip by sometimes, most of the times the ratings are good enough for you to quicksearch on wfm and riven.market for deals by automating the searches.

Something so simple took quite some time for me as a programming beginner, if you actually wanted to focus on only giving a price, instead of a performance rating, it'd be quite difficult as you'd have to generate your own statistics for each weapon and combination of stats from third party websites such as wfm, riven.market or rivenhunter.com, and you'd see the listings, but not the sells, also the information can be erroneus, be it by mistakes or maliciously, it's not rare to find people manipulating market prices.

The only information you can trust is provided by WF API, although it's quite generic and I don't even know if it's functioning anymore.

In any case, I wish you good luck with your pricing methods, and, if you want to share your approach, feel free to reply here as I'd love to hear about it.