Apify CLI: run Actors locally from your terminal

$ echo '{"searchStringsArray": ["Apify"] }' |
apify call compass/crawler-google-places --silent --output-dataset
[{
"searchString": "Apify",
"categoryName": "Corporate office",
"address": "Vodickova 704/36, 110 00 Nove Mesto, Czechia",
"street": "Vodickova 704/36",
"city": "New Town",
"postalCode": "110 00",
"countryCode": "CZ",
"website": "https://apify.com/"
"location": {
  "lat": 50.0813695,
  "lng": 14.4254026
},
"locatedIn": "Lucerna Passage",
"plusCode": "3CJG+G5 Prague, Czechia",
"rank": 1,
"title": "Apify",
"isAdvertisement": false,
...
}]

In his essay, In the Beginning… Was the Command Line, Neal Stephenson wrote, "The command line interface is not just a lingua franca but a form of magical incantation." At Apify, we're bringing some of that command-line magic to web automation. We're introducing a significant upgrade to our Command Line Interface (CLI) functionality, showing that in web automation and scraping, the command line is as relevant as ever.

New features for workflow improvements

$ apify call tri_angle/airbnb-scraper \ 
--input='{"locationQueries":["San Francisco"]}' 

With this update, you can run pre-built web automation and web scraping scripts (we call them Actors) using local parameters without leaving your terminal. Despite running remotely on our infrastructure, the command executes as if it were a local script. This eliminates the need to visit Apify Console for passing inputs, improving CLI workflow for Actor interactions.

Who benefits and key advantages

This feature is primarily designed for command-line natives, Unix-minded individuals, and professionals in related fields such as data analysis and DevOps. Anyone who prefers working with the command line will find this update valuable.

The core benefit of this update is its ability to bring complex web automation and data extraction to shell scripting. While shell scripting has long been synonymous with easy automation in general computing tasks, this update reverses the flow. By enabling shell-script control of sophisticated web processes, Apify is bridging the gap between traditional automation and advanced web-based tasks. This integration of powerful web capabilities into familiar command-line workflows is what makes this update exciting.

Using Apify CLI with other Unix tools

We're pleased to announce that we've resolved a key limitation mentioned in our initial release. The issue of logs being mixed with results has been addressed. Now, by appending -s -o (short for --silent --output-dataset) to the command, you can get a clean output without logs, making it much more suitable for video demonstrations or situations where you need just the content of the result dataset.

For example:

$ echo '{"searchStringsArray": ["Apify"] }' | 
apify call compass/crawler-google-places -s -o | 
jq -r .[0].address 

Vodickova 704/36, 110 00 Nove Mesto, Czechia

This command will only return the dataset in JSON to stdout, without any accompanying logs. You can then integrate it via Unix pipes with all the powerful command-line tools in your tool belt, such as jq for JSON processing.

Current limitations and future improvements

However, there are still some things we're working on:

• Currently, retrieving key-value store items for the Actor run requires multiple commands.
• For larger datasets, filtering the output from the apify call command is not currently supported as in the API. This functionality will be the subject of the future apify dataset CLI namespace.

Ready to get the most out of Apify CLI?

For detailed information about this feature, including usage examples and the latest updates like the --silent and --output-dataset options for clean output, visit our CLI docs.

Please check out these new CLI features and give us your feedback. We're always happy to hear about how you use Apify. It helps us understand what kinds of features and improvements we should make in the future.