We're excited to announce dynamic Actor memory, a new feature that allows Actor developers to define expressions that automatically calculate the right amount of memory for each run. These expressions use the actual input and run options to determine optimal memory allocation before the run starts.
Think of it as a short script that says, "If the user provides 10 URLs, allocate 512 MB; if they provide 1,000 URLs, allocate 4 GB." Memory allocation gets calculated once before execution begins and stays constant throughout the run.
Why dynamic memory?
Actor developers face a difficult trade-off when setting default memory:
- Set it too low, and runs become slow
- Set it too high, and users pay for unused resources on smaller runs
- Leave it to users to configure, and you add unnecessary complexity
An Actor processing 10 product pages needs different resources than one processing 1,000 pages. Until now, developers had to pick a single default value and hope it worked reasonably well across all use cases.
Dynamic Actor memory solves this. Developers define an expression that calculates memory before each run starts based on the input. This means users automatically get the right resources for their specific job without manual configuration. Small jobs cost less, and large jobs get the memory they need to run faster. And if users want to override the calculation, they can still set memory manually via the UI or API.
Real-world examples
Dynamic Actor memory works behind the scenes to make sure each run gets the right amount of memory.
To enable dynamic memory, the developer provides an expression in the Actor configuration (actor.json).
Below are two examples that show how dynamic memory behaves in common real-world scenarios.
Example 1: scaling memory by number of URLs
Let’s say your Actor accepts a list of product URLs to scrape. This is the expression that would allocate memory based on how many URLs the user has requested:
get(input, 'startUrls.length', 1) * 64
This expression reads the number of URLs from the input and multiplies it by 64 MB. If startUrls is missing or empty, it defaults to 1.
For example, if a user provides 5 URLs, the calculation would be: 5 × 64 = 320 MB, rounded to the nearest power of 2 (256 MB).
Example 2: scaling memory based on whether users also want to scrape product reviews
Some Actors need more complex expressions to cover multiple use cases. For example, scraping product reviews requires more memory than scraping product details alone.
urlsCount = get(input, 'startUrls.length', 0);
reviewsMultiplier = max(get(input, 'maxReviews', 1) / 10, 1);
urlsCount * reviewsMultiplier * 128
This expression uses intermediate variables to calculate memory based on both the number of URLs and how many reviews to scrape per product.
For example, if a user provides 10 URLs and wants 5 reviews per product, the calculation would be: 10 × (5/10) × 128 = 640 MB (rounded to 512 MB).
Conclusion
Dynamic Actor memory makes Actors smarter. If your Actor processes variable inputs or serves users with different workloads, enabling dynamic memory improves performance and user experience.
For complete documentation on expression syntax, memory limits, best practices, and testing expressions, check out the dynamic Actor memory docs.
