Egypt Electronics Price Scraper

Search a product across multiple Egyptian electronics stores at once, instead of relying on Google (whose SEO these stores mostly don't rank for).

Setup

pip install -r requirements.txt

Desktop app

The friendliest way to use it — a small cross-platform window (Windows / macOS / Linux):

python app.py

It has exactly three things:

• A search bar — type a product name and press Enter (or click Search). It matches broadly, so a phrase like esp32 devkit finds anything containing those words.
• An update bar across the top — on launch it checks this GitHub repo. If a newer version exists, an "Update now" button appears; clicking it downloads the latest code, applies it in place, and automatically restarts the app (no manual reinstall). If you're up to date, the bar is green. Since you're already online to scrape, the check is free.
• A terminal-style progress log — streams each store's result as it arrives, exactly like the command line, with failed stores flagged.

It deliberately does not show the product list in the window. Results are written to an Excel file (with thumbnails and out-of-stock highlighting, same as the CLI). By default they save to a Egypt Electronics Scraper folder in your home directory, but the "Save location" button (folder icon) lets you pick any folder, and your choice is remembered between runs. The footer has an "Open results folder" button.

Command-line usage

python main.py "arduino uno"
python main.py "ga25-370 motor encoder" --out results

Prints a price-sorted comparison table to the console and saves a timestamped .xlsx file to results/, with each row's product thumbnail embedded directly in the "Image" column (downloaded from the store's image URL). The URL column is also a clickable hyperlink.

Out-of-stock products are kept in the results (not filtered out) but highlighted so they're easy to spot: red text in the console table, and a red-filled row in the spreadsheet.

Every run gets its own file — the filename includes a microsecond-precision timestamp, and a numeric suffix is added in the rare case of a collision, so a new search never overwrites a previous one.

Why a store sometimes fails

A store can fail for two different reasons, shown differently:

• Transient server error (500/502/503) — the session automatically retries twice with a short backoff before giving up, so a one-off blip on the store's end usually resolves itself without you noticing.
• Rate limiting (429) — this is the store deliberately throttling us, so it is not retried automatically (some sites ask you to wait 30+ minutes via a Retry-After header, which would stall the whole search). If the store sends that header, it's now shown in the error message; if it doesn't, the store just isn't telling us how long the block lasts.

If the same store keeps returning 429 well after a Retry-After-style wait, it's likely an IP-level anti-bot block (common on WordPress.com/Automattic-hosted stores like Makers Electronics) rather than a simple per-minute limit — those can last much longer than a few minutes, especially after many test queries in a short window. Spacing out searches (rather than running several back-to-back) is the practical mitigation; there's no reliable way to shorten the block from our side.

Included stores (16)

Stores fall into two groups by how we get their data.

Group A — official JSON API (fast, robust). These expose a free, unauthenticated JSON search endpoint, verified by the actual Content-Type/JSON shape of the response (not just the HTTP status code — a couple of the excluded sites return 200 OK with an HTML shell for any path, which looks like a match on status code alone but isn't). Adapters live in adapters.py.

• Shopify (/products.json): Circuits Electronics, DevBoards Market, FUT Electronics
• WooCommerce (/wp-json/wc/store/v1/products): Free Electronic, UGE-One, Tech Hub EG, Flux Electronix, Most Electronic, Ampere Electronics, Makers Electronics, 3Duino

Group B — no public API (scraped per-site). These need a store-specific approach; adapters live in html_adapters.py. HTML scraping is inherently more fragile than an API — if a store restyles its product cards, its adapter's CSS selectors may need updating.

Store	Platform	How
Electra Store	Laravel Livewire	Parse server-rendered HTML from /products?q= (product cards are div.group.relative)
Filotronix	PrestaShop	Parse HTML from /search?controller=search&s= (article.product-miniature)
RAM Electronics	Odoo eCommerce	Parse HTML from /shop?search= (form.oe_product_cart)
MTM Electronic	Next.js + Laravel API	Call the private JSON API (test.mtm-electronic.com/public/api/products?search=) found in the site's frontend bundle
Electrolik	WooCommerce (WoodMart theme)	Store API is blocked (403), so parse the front-end search HTML (div.product-grid-item); prices are European-formatted (1.250,00)

Still excluded (not implemented)

Re-checked as of the latest round — all still blocked or without a usable data source:

Store	Platform	Why excluded
it-tronic.store	Unknown	Cloudflare JS challenge (HTTP 403 "Just a moment") blocks plain HTTP requests; needs a headless browser or Cloudflare-bypass
microohm-eg.com	Unknown	Cloudflare JS challenge (same)
www.maamoon.com	Wix	Client-rendered SPA with no stable public API; needs a headless browser
el-gammal.com	Supabase backend	Every path returns the React SPA shell; data comes from a Supabase project that would need its tables/RPCs reverse-engineered
lampatronics.com	Custom (Laravel)	Every path (incl. /products.json) returns the Laravel HTML shell; no product API endpoint found
mecha-tronx.com	—	Still shows a "coming soon / under maintenance" page (HTTP 503), no live products

Releasing a new version (for the maintainer)

The desktop app's update bar compares its built-in version against the VERSION file on this repo's main branch. To publish an update:

1. Make your changes (e.g. fix a broken scraper selector).
2. Bump the version in two places, keeping them identical: APP_VERSION in version.py and the VERSION file.
3. Commit and push to main.

Anyone who opens the app now sees "Update available" and an "Update now" button. Clicking it downloads main as a ZIP, overwrites the app's files (skipping results/ and __pycache__/), best-effort runs pip install -r requirements.txt for any new dependencies, and restarts. See updater.py.

Two things to keep in mind:

• The update mechanism itself must be on main for the feature to work. updater.py and the update-button version of app.py need to be pushed before any user can self-update. (After that, updates just flow.)
• Don't click "Update now" on your own development checkout. It overwrites local files with whatever is on main, which would clobber uncommitted work. It's meant for end users running a clean copy.
• This works for the run-from-source distribution. A packaged .exe can't overwrite its own running binary, so it would need a different updater (download new exe → swap on next launch).

Packaging into a standalone app (optional)

To give people an app they can run without installing Python, build a bundle on each target OS (you can't cross-build):

flet pack app.py --name "Egypt Electronics Scraper"

This produces a single executable (.exe on Windows, .app on macOS). Two caveats: it must be built on the OS you're targeting, and an unsigned build triggers a SmartScreen/Gatekeeper "unknown publisher" warning (and occasionally an antivirus false positive) until you code-sign it.

Adding a new store

1. Add an entry to stores_config.py with name, base_url, and platform.
2. If it's Shopify or WooCommerce, that's it — the existing adapter handles it.
3. Otherwise, write a new search_<platform>(store_name, base_url, query, session) -> list[Product] function (in adapters.py for a JSON API, or html_adapters.py for HTML scraping) and register it in the ADAPTERS dict in adapters.py.