Skip to content
Get packs

Guide · Open source

How to use ar-cutpaste: setup, Photoshop and local server

Updated · written with AI from the project's own documentation and checked against our catalogue data

ar-cutpaste is an open-source research prototype that uses a phone camera and machine learning to pick an object out of your surroundings and paste it into Photoshop. This guide explains how its three parts fit together and what you need before you start. By the end you will know the setup steps the README describes and where to find the detailed instructions for each module.

AR Cut & Pastecyrildiagne/ar-cutpaste · ★ 14.6k · MIT

Quick start

  1. 1Clone the repository. Run git clone on the ar-cutpaste repository to get the source code.
  2. 2Enable Remote Connection in Photoshop. Go to Preferences > Plug-ins, enable Remote Connection and set a password.
  3. 3Match Photoshop document settings. Make sure your document settings match server/src/ps.py and that the document has a non-blank background.
  4. 4Choose a BASNet service. Either deploy your own BASNet service with a CUDA GPU or use the community endpoint with --basnet_service_ip.
  5. 5Run the local server. Follow the /server folder instructions to set up and run the local server, using pip with requirements.txt.
  6. 6Deploy the mobile app. Follow the /app folder instructions to deploy the app to your phone.

What ar-cutpaste is and how it works

ar-cutpaste is an AR and machine learning prototype. You point a phone camera at something, and it cuts that element out and pastes it into an image editing program. The README says only Photoshop is handled at the moment, though outputs may support other programs in the future.

It runs as three independent modules: a mobile app, a local server that bridges the app and Photoshop, and an external object detection and background removal service.

  • Mobile app: deployed to your phone (see the /app folder).
  • Local server: links the app to Photoshop and finds the screen position pointed at by the camera using screenpoint.
  • Object detection service: removes the background using the BASNet model, either self-hosted or via a community endpoint.

Check the prerequisites before you start

The README lists a few requirements. You need Photoshop with Remote Connection enabled, and Python, because the local server installs its dependencies with pip from requirements.txt. A CUDA GPU is only needed if you host the BASNet service yourself.

  • Photoshop with Remote Connection enabled.
  • Python, for the local server.
  • A CUDA GPU, only if you self-host the BASNet service.
  • Git, to clone the repository.

Clone the repository and configure Photoshop

Start by cloning the repository. Then open Photoshop and set it up so the local server can talk to it.

In Photoshop, go to Preferences > Plug-ins, enable Remote Connection and set a password you will remember, because the local server needs it. Next, make sure your document settings match those in server/src/ps.py, otherwise only an empty layer will be pasted. Also give the document some background, since a blank background may not give SIFT enough features to match.

  • Go to Preferences > Plug-ins and enable Remote Connection.
  • Set a friendly password for the connection.
  • Match your document settings to server/src/ps.py.
  • Use a document with a background that is not blank.
git clone https://github.com/cyrildiagne/ar-cutpaste

Choose a BASNet service option

The object detection step needs a BASNet service. The README offers two options. The first is to deploy your own model using the BASNet-HTTP wrapper, which requires a CUDA GPU. The README says you must then configure the deployed service URL in the local server, and use a different port if BASNet runs on the same computer as the local service.

The second option is a community-provided public endpoint, which is useful if you have no CUDA GPU. The README gives no uptime guarantee for it.

  • Option 1: self-host BASNet via the basnet-http wrapper (requires a CUDA GPU).
  • Option 2: use the community endpoint by launching the local server with --basnet_service_ip set to the URL given in the README.
--basnet_service_ip http://u2net-predictor.tenant-compass.global.coreweave.com

Set up and run the local server and mobile app

Follow the instructions in the /server folder to set up and run the local server. The README's older setup block used a virtualenv, activation, and pip install -r requirements.txt, but that block is commented out in the README, so check /server for the current steps.

Then follow the instructions in the /app folder to deploy the mobile app to your phone.

  • Set up and run the local server from the /server folder.
  • Deploy the mobile app from the /app folder.
pip install -r requirements.txt

What people use it for

  • Exploring how AR, image segmentation and Photoshop automation can be combined in one pipeline.
  • Testing salient object detection with the BASNet model in a practical setup.
  • Prototyping a camera-to-Photoshop workflow for research purposes.

Frequently asked questions

Is ar-cutpaste free to use?

ar-cutpaste is released under the MIT licence, and its source is on GitHub. The README does not list any prices. Check the repository's licence file for the full terms.

What do I need to run ar-cutpaste?

You need Photoshop with Remote Connection enabled, Python for the local server, and a phone to run the mobile app. A CUDA GPU is only required if you host the BASNet service yourself.

Does ar-cutpaste work with editors other than Photoshop?

No. The README says only Photoshop is handled currently, and that other outputs may be supported in the future.

Is there an easier app based on this research?

Yes. The README points to clipdrop.co as an easy-to-use app based on this research. It also states that ar-cutpaste itself is a research prototype and not a consumer tool.

Why does my paste produce only an empty layer?

The README says this happens when your Photoshop document settings do not match those in server/src/ps.py. Match the settings and make sure the document has a background.

What are the alternatives to ar-cutpaste?

The source lists comparable projects: clipdrop, screenpoint and tflite-react-native. screenpoint is used by ar-cutpaste itself to find the screen position pointed at by the camera.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

ar-cutpaste suits developers and researchers who already use Photoshop and can run a local server, and who want to study AR with BASNet in a working pipeline. If you just want a ready-made tool to cut and paste objects, the README points you to clipdrop.co instead.

Install options & downloads