RELIXA

Try Relixa

This tutorial finds the same products in two stores’ catalogs. It uses Sparkly Lite for blocking and Iris for matching. The Abt–Buy sample dataset is included in both tools’ repositories, in the data/abt_buy folder. Complete steps 01–04 to produce a file of predicted matches. Step 05 is optional and shows how Iris collects training labels.

For an introduction to blocking, matching, and these tools, see the Products page.

Before you begin

Use an Apple silicon Mac (M1 or later) with macOS 11 or later, Git, an internet connection, and Python 3.10, 3.11, or 3.12. This walkthrough runs both tools on the same Mac.

Open Terminal from Applications → Utilities → Terminal.

Run the following commands in Terminal:

bash
uname -m
sw_vers -productVersion
python3 --version
git --version

These commands check your Mac’s processor, macOS version, Python version, and Git installation. They should report arm64, macOS 11 or later, Python 3.10–3.12, and a Git version. If Git asks to install the Command Line Tools, complete that installation and rerun git --version.

Python missing or unsupported?

If python3 is missing or reports a version outside 3.10–3.12, use one of the two installation methods below. If it already reports a supported version, continue to step 01.

Method 1: Python.org

Download the macOS 64-bit universal2 installer from the Python 3.12 download page and run the downloaded .pkg file. Follow the installer prompts. Then run the following commands in Terminal:

bash
export PATH="/Library/Frameworks/Python.framework/Versions/3.12/bin:$PATH"
python3 --version

The export PATH command makes this Python installation available as python3 in the current Terminal window. The second command prints its version.

Method 2: Homebrew

If Homebrew is already installed, run the following commands in Terminal:

bash
brew install python@3.12
export PATH="$(brew --prefix python@3.12)/libexec/bin:$PATH"
python3 --version

The first command installs Python 3.12. The export PATH command makes it available as python3 in this Terminal window, and the final command prints its version. Specifying python@3.12 avoids installing a newer version that Iris does not support. See the Homebrew Python 3.12 formula for package details.

After either method, the final command should print Python 3.12 followed by a patch version, such as Python 3.12.10. Continue to step 01 in the same Terminal window.

  1. Install Sparkly Lite

    Run the following commands in Terminal:

    bash
    mkdir -p ~/relixa-demo
cd ~/relixa-demo
test -d sparkly_lite || git clone https://github.com/relixa-ai/sparkly_lite.git
cd sparkly_lite
bash install.sh

    These commands create ~/relixa-demo inside your home folder and download Sparkly Lite into it. If its folder already exists from an earlier run, it is reused.

    The installer creates .venv inside ~/relixa-demo/sparkly_lite, installs Sparkly Lite and its dependencies there, and tests the engine. Wait for Installation complete. before continuing.

    Run the following commands in the same Terminal window:

    bash
    source .venv/bin/activate
command -v sparkly-lite

    The first command activates Sparkly Lite’s Python environment. The second shows where the sparkly-lite command is installed. Its output should end in /relixa-demo/sparkly_lite/.venv/bin/sparkly-lite.

  2. Generate candidate pairs

    Run the following commands in the same Terminal window:

    bash
    cd ~/relixa-demo/sparkly_lite
source .venv/bin/activate
sparkly-lite block data/abt_buy/table_a.parquet data/abt_buy/table_b.parquet \
    --columns name,description --k 20 --out candidates.parquet

    These commands enter the sparkly_lite folder, activate its Python environment, and run blocking on the included Abt–Buy tables. The tables contain 1,081 Abt products and 1,092 Buy products.

    For each Buy product, this command selects the 20 closest Abt products by name and description. It writes the candidate pairs to candidates.parquet in the sparkly_lite folder. The output should include the following. Runtime may vary depending on your machine:

    output
    Dataset           abt_buy
|A|               1,081
|B|               1,092
Blocking Columns  name, description
K                 20
Candidates        21,721
Reduction         98.1599%
Time              66 ms
Saved             candidates.parquet (21,721 rows)

    Confirm that the final line reports Saved candidates.parquet (21,721 rows). Each row holds the IDs of one Abt product and one Buy product.

    Blocking reduces 1,180,452 possible pairs to 21,721 candidates for Iris to examine in the matching step.

  3. Install Iris

    Run the following commands in the same Terminal window:

    bash
    deactivate
cd ~/relixa-demo
test -d iris || git clone https://github.com/relixa-ai/iris.git
cd iris
PYTHON=python3 bash install.sh

    These commands deactivate Sparkly Lite’s environment, return to ~/relixa-demo, and download Iris alongside Sparkly Lite. An existing iris folder is reused. The last command runs Iris’s installer.

    The installer creates a separate .venv inside the iris folder and tests Iris. Wait for OK: iris works on this machine. before continuing.

    Run the following commands in the same Terminal window:

    bash
    source .venv/bin/activate
cp ../sparkly_lite/candidates.parquet .
ls candidates.parquet

    The first command activates Iris’s environment. The second copies candidates.parquet from Sparkly Lite into the iris folder. The final command checks that the copied file is present and should print candidates.parquet. Iris includes the same sample tables in data/abt_buy.

  4. Train and apply a matcher

    Run the following commands in the same Terminal window:

    bash
    cd ~/relixa-demo/iris
nano match.py

    These commands enter the iris folder and open match.py in the built-in terminal text editor. The editor creates the file when you save it.

    Click the Copy icon on the Python snippet below, then paste it into the editor in Terminal:

    match.py
    from xgboost import XGBClassifier
import pandas as pd
import iris

table_a = iris.read_table('data/abt_buy/table_a.parquet')
table_b = iris.read_table('data/abt_buy/table_b.parquet')
candidates = iris.read_pairs('candidates.parquet')
labeled_pairs = iris.read_pairs('data/abt_buy/labeled_pairs.parquet', with_label=True)

columns = ['name', 'description', 'price']
features = iris.create_features(table_a, table_b, columns, columns)

labeled = iris.featurize(features, table_a, table_b, labeled_pairs[['id1', 'id2']])
labeled = labeled.merge(labeled_pairs[['id1', 'id2', 'label']], on=['id1', 'id2'])
model = iris.SKLearnModel(XGBClassifier, nan_fill=0.0)
matcher = iris.train_matcher(model, labeled)

fvs = iris.featurize(features, table_a, table_b, candidates)
predictions = iris.apply_matcher(matcher, fvs, 'feature_vectors', 'prediction', 'confidence')
iris.save_predictions(predictions, 'matches.parquet')

matches = pd.read_parquet('matches.parquet')
matches['abt_name'] = matches['id1'].map(table_a.set_index('_id')['name'])
matches['buy_name'] = matches['id2'].map(table_b.set_index('_id')['name'])
matches[['abt_name', 'buy_name', 'probability']].to_csv('matches.csv', index=False, float_format='%.3f')

gold = iris.read_pairs('data/abt_buy/gold.parquet')
metrics = iris.evaluate(predictions, gold)
print(f"precision {metrics['precision']:.4f}  recall {metrics['recall']:.4f}  f1 {metrics['f1']:.4f}")

    Press Control+O, then Return to save match.py. Press Control+X to return to the terminal prompt.

    Run the following command in Terminal:

    bash
    python match.py

    Running match.py reads the sample’s 550 labeled pairs (249 matches and 301 non-matches), trains a matcher, scores the 21,721 candidates, and exports the predicted matches.

    The final line should be close to the following. Small numerical differences may occur across machines:

    output
    precision 0.9665  recall 0.8687  f1 0.9150
    How the script works
    • create_features selects comparison methods for each column. featurize converts pairs into feature vectors.
    • train_matcher learns from the labeled pairs. apply_matcher scores all 21,721 candidates.
    • save_predictions writes 986 pairs with a match probability of at least 0.5 to matches.parquet. The next four lines export the product names and probabilities to matches.csv.
    • The final three lines compare predictions with the sample’s reference answers in gold.parquet, which contains 1,097 known matches.
    • Precision: 953 of 986 predicted matches are correct (96.65%).
    • Recall: 953 of 1,097 known matches are found (86.87%). The total includes the 10 excluded by blocking.
    • F1: the combined precision and recall score is 0.9150.

    Run the following command in Terminal:

    bash
    open matches.csv

    Open matches.csv in a spreadsheet application, such as Numbers or Excel.

    Each CSV row contains the Abt product name, the Buy product name, and a match probability. The preview below shows only the product names from six of the 986 predicted matches:

    matches.csvProduct names · 6 of 986 rows
    abt_namebuy_name
    sony active style headphones in black mdras50gsport neckband earbuds mdr as50g
    apple macbook air superdrive mb397gaapple 8x dvdrw drive mb397g/a
    alpine 2-din dvd/cd/mp3 / wma receiver/av head unit ivaw205alpine iva-w205 car video player
    lg wm0642hw 4.0 cu ft white rear control washer wm0642hwhlg electronics wm0642hw front load washing machine with 7 washing programs
    panasonic all-in-one flatbed laser fax kxflb801panasonic kx-flb801 multifunction printer kx-flb801
    belkin cush top for computer laptop f8n044orgbelkin cushtop f8n044-org

    These six pairs are confirmed matches despite their different names. The full sample output also contains 33 false matches, including different wall-mount colors (vm400b and vm400s), 8 GB and 16 GB drives, and phones paired with add-on handsets. To check a predicted match, compare the product names and model numbers, including details such as color and capacity.

    You have now completed blocking and matching. You can finish here, or continue to step 05 to see how Iris selects examples for labeling.

  5. Don’t have labeled data?

    The previous example trained a matcher using labeled pairs included with the sample. This example starts without that labeled-pairs file. It uses the sample’s reference answers to demonstrate collecting labels automatically, so you do not need to label pairs during this run.

    Run the following commands in the same Terminal window:

    bash
    cd ~/relixa-demo/iris
nano match_active.py

    These commands open a separate file, match_active.py, in the iris folder. The script from step 04 stays available as match.py.

    Click the Copy icon on the Python snippet below and paste it into the editor in Terminal:

    match_active.py
    from xgboost import XGBClassifier
import pandas as pd
import iris

table_a = iris.read_table('data/abt_buy/table_a.parquet')
table_b = iris.read_table('data/abt_buy/table_b.parquet')
candidates = iris.read_pairs('candidates.parquet')
gold = iris.read_pairs('data/abt_buy/gold.parquet')

columns = ['name', 'description', 'price']
features = iris.create_features(table_a, table_b, columns, columns)

fvs = iris.featurize(features, table_a, table_b, candidates)
labeler = iris.GoldLabeler(gold)
seeds = iris.create_seeds(fvs, 50, labeler, checkpoint_path='labels.parquet')
model = iris.SKLearnModel(XGBClassifier, nan_fill=0.0)
labeled = iris.label_al(model, 'batch', labeler, fvs, seeds,
                        checkpoint_path='labels.parquet', batch_size=10, max_iter=50)
matcher = iris.train_matcher(model, labeled)
predictions = iris.apply_matcher(matcher, fvs, 'feature_vectors', 'prediction', 'confidence')
iris.save_predictions(predictions, 'matches.parquet')

matches = pd.read_parquet('matches.parquet')
matches['abt_name'] = matches['id1'].map(table_a.set_index('_id')['name'])
matches['buy_name'] = matches['id2'].map(table_b.set_index('_id')['name'])
matches[['abt_name', 'buy_name', 'probability']].to_csv('matches.csv', index=False, float_format='%.3f')

gold = iris.read_pairs('data/abt_buy/gold.parquet')
metrics = iris.evaluate(predictions, gold)
print(f"precision {metrics['precision']:.4f}  recall {metrics['recall']:.4f}  f1 {metrics['f1']:.4f}")

    Press Control+O, then Return to save match_active.py. Press Control+X to return to the terminal prompt.

    Run the following command in Terminal:

    bash
    python match_active.py

    Running match_active.py collects 50 initial labels, followed by 10 labels in each of 50 rounds. After each batch, Iris retrains the matcher and selects the pairs it is least certain about. This process is called active learning. The reference answers in gold.parquet supply the 550 labels in this demonstration.

    The run saves the collected labels in labels.parquet and replaces matches.parquet and matches.csv with the new predictions. The labels supplied in step 04 were collected by this process, so the results should be close to:

    output
    precision 0.9665  recall 0.8687  f1 0.9150

    Iris saves the collected labels in labels.parquet and reuses them on later runs.

    Start a new labeling session

    Run the following command in Terminal:

    bash
    nano match_active.py

    In both create_seeds and label_al, replace checkpoint_path='labels.parquet' with checkpoint_path='labels-new.parquet'. Press Control+O and Return to save, then Control+X to exit.

    Run the following command in Terminal:

    bash
    python match_active.py

    Iris collects a new set of labels in labels-new.parquet. The labels from the previous session remain in labels.parquet. Choose a filename that does not already exist for each new session.

Next steps

  • Apply existing blocking rules

    Delex Lite applies rules to include or exclude pairs on your Mac and exports candidates compatible with Iris.

    Explore Delex

  • Scale beyond 3 million rows per table

    Sparkly Lite supports up to 3 million rows per table. Sparkly Plus supports tens of millions on your Mac, using the same commands and files.

    Explore Sparkly Plus

  • Plan an enterprise deployment

    Pro runs on a server, on premises or in your cloud. Contact us to discuss workloads with hundreds of millions of rows and contractual requirements.

    Request a demo