Core concepts

Intent-based testing

Most testing tools rely on UI selectors: find this button by its ID, tap it, and check a specific element appears. This is fast to execute, and works well until someone redesigns a screen, and then every test built on those selectors fails.

Semaloop works differently. Instead, you describe how functionality should behave, and our agent figures out how to do it: “log in and confirm the dashboard loads”. This means that when you ship a redesign, a new feature, or a simple update, there’s nothing you need to update in Semaloop, saving you time and effort.

Agents

An agent is the thing that runs your test. It reads your intent, decides what actions to take on the device, and reports back what happened.

Agents aren’t scripts. They interpret the state of the app by looking at the screen, and work out the steps needed to fulfil the test’s intent, adjusting as they go.

Devices: real vs simulator

On iOS you can run tests on a real device or in a simulator, and each has its place. On Android, tests run on emulators today; the comparison below is about iOS.

Real devices give you the full picture: hardware performance, network conditions, push notifications, audio input, and much more. They’re best used when you need confidence that your app behaves correctly in the real world. Semaloop makes using real devices easy through its TestFlight integration.

Simulators are great for the development of your app. However, they have less functionality than real devices. We often see customers integrating with simulators much earlier in their development process, where they’ll ship a pre-release build to Semaloop through their CI/CD process and have it run on a simulator.

Here’s a quick overview of the differences between real devices and simulators in Semaloop:

	Real device	Simulator
TestFlight builds	Yes	No
CI/CD integration	Yes. Requires an .ipa signed with Semaloop device UDIDs.	Yes. Just send us an .app file.
Secure auth (Face ID/Touch ID)	No	Yes
Audio input	Yes	No
Camera	Yes	No
HomeKit	Yes	No
Push notifications	Yes	No
Accurate display rendering	Yes	No. Text and images can look jagged depending on your Mac’s resolution.
Production-matching filesystem	Yes. Case-sensitive, matching production.	No. Simulators use a case-insensitive filesystem, which can hide real bugs.
Realistic performance and network conditions	Yes	No. Simulators share the networking stack with their host, rather than device-equivalent constraints.

It’s worth noting that Semaloop supports both, at any stage of development.

The choice is up to you, but we find most Semaloop customers are running on real devices, because it’s what gives them the most confidence that they’re shipping something that works in the real world.

On Android, tests run on emulators, so the hardware-dependent rows above — audio input, camera, real network conditions — do not apply yet. Android real-device coverage is in progress.

Tests and runs

A test is the definition of intended behaviour, plus any setup or tools it needs. A collection of tests is a test suite.

A run is an execution of a test, against a given build. You’ll write a test once and run it many times, and each run produces its own result, so you can track a test’s behaviour over several builds.

Active tests are those that will be automatically run whenever you push a new build to Semaloop. Draft tests are still work in progress, and won’t be included in the next test suite run.

Issues

Not every problem an agent finds leads to a failed test. Sometimes a test technically passes, but the agent hit friction along the way: a confusing button, a slow load, or some contrast that didn’t feel quite right. Our agents surface these as issues alongside a test success or failure.

See the issues documentation.