This post explains how iOS Universal Links work and what is required to implement them. It covers how to handle incoming links in an iOS app, how to create and configure the Apple App Site Association (AASA) file, and how to correctly host the file so that iOS can securely associate a domain with an app and route matching URLs from the web into the app.

Who should read the post?

• Software Developers who plan to integrate this feature into their app.

• People in tech with little to no experience on Universal Links who want to learn something new.

Introduction

Have you ever tapped on an ad or email link and been taken directly to the desired application without seeing a website or pop-up asking for permission? That seamless transition from the web to the app isn't magic; it's powered by Universal Links.

Universal Links allow iOS (or Android) to open your app automatically when a user taps a specific URL belonging to a domain you control. If the app is installed, the link opens the app. If it isn't, the same link simply opens in the browser. This behaviour becomes a powerful tool for ads, emails, deep navigation, onboarding flows, and the overall user experience.

Despite how seamless they feel to users, it requires careful configuration across your iOS app and backend. For developers encountering them for the first time, the setup process can be surprisingly strict and easy to misconfigure.

This post will cover the steps required to make Universal Links work on iOS.

• Configure your iOS app to handle incoming links.

• Creating the Apple App Site Association (AASA) file.

• Hosting the AASA file under the /.well-known path.

By the end, you will have a better understanding of how to set up an application to support Universal Links, which will give you a head start if you plan on integrating this feature into your software.

Configuring your iOS app to handle incoming links

The first step is ensuring that your iOS app knows how to react when someone taps a Universal Link. Although Universal Links originate on the web, iOS redirects them to the app via the App Delegate. This makes the App Delegate the central point where your app decides how to handle an incoming URL. Options include opening a specific screen, triggering navigation, or passing parameters deeper into your React Native layer.

When a Universal Link is activated, iOS calls two native methods:

• scene(_:continue:) for apps using the modern Scenes API

• application(_:continue:restorationHandler:) for older setups

Your job is to implement at least one of these and route the received URL into your code.

Here’s a simplified example using the SceneDelegate, which is common in React Native projects:

// SceneDelegate.swift
func scene(_ scene: UIScene, continue userActivity: NSUserActivity) {
    guard userActivity.activityType == NSUserActivityTypeBrowsingWeb,
          let incomingURL = userActivity.webpageURL else { return }

    // Pass the link to React Native
    RCTLinkingManager.application(UIApplication.shared,
                                  open: incomingURL,
                                  options: [:])
}

If your project still relies on the AppDelegate, the equivalent version looks like:

// AppDelegate.m (Objective-C)
- (BOOL)application:(UIApplication *)application
            continueUserActivity:(NSUserActivity *)userActivity
              restorationHandler:(void (^)(NSArray *))restorationHandler
{
  return [RCTLinkingManager application:application
                      continueUserActivity:userActivity
                        restorationHandler:restorationHandler];
}

This implementation ensures your React Native app receives the link through the Linking API, where you can listen for it:

import { Linking } from 'react-native';

Linking.addEventListener('url', ({ url }) => {
  // Handle navigation here
});

At this point, the native layer is ready — your app can now receive Universal Links.
Next, we’ll prepare the website side of the handshake by creating the AASA file so iOS knows your app is authorised to handle those links.

Creating the Apple App Site Association (AASA) File

Now that the app can receive Universal Links, the next step is to convince iOS that your domain is allowed to open your app. This is where the AASA file comes in.

At a high level, the AASA file is a JSON document on your website that tells iOS:

• which app(s) belong to your domain

• which URL paths those apps are allowed to handle

• whether you support features like Shared Web Credentials or Universal Links

When a user installs your app, iOS automatically fetches the file from your domain. If the file is valid and correctly formatted and located in the expected place, iOS registers your app as the handler for all matching paths.

AASA File Structure

Below is the minimal structure iOS expects in an apple-app-site-association file and how each section influences link handling:

{
  "applinks": {
    "details": [
      {
        "appIDs": ["ABCDE12345.com.example.app"],
        "paths": ["/products/*", "/profile/*"]
      }
    ]
  }
}

The paths are obviously the paths from the app that should trigger the universal link behaviour.

The appID is formed by: <Application Identifier Prefix>.<Bundle Identifier>.
The Application Identifier Prefix is visible in the Apple Developer portal under Membership or in Xcode by choosing your project → Signing & Capabilities → selecting your team. The Bundle Identifier comes from the same Xcode section and uniquely identifies your app (e.g., com.example.app).

Add the Associated Domains Entitlement to Your App

iOS will only validate the AASA file if your app explicitly declares the domain associated with it. To do this, enable the Associated Domains capability in Xcode and add an entry such as:

applinks:<your-domain>.com

This links your app to the domain so iOS knows where to retrieve and verify your AASA file.

Hosting the AASA File under the /.well-known path.

For iOS to recognise your Universal Links, the AASA file must be publicly accessible at:

https://your-domain.com/.well-known/apple-app-site-association

iOS automatically fetches this file during app installation to verify the domain-app association. Hosting it in any other path or behind redirects will cause verification to fail, so serving it correctly is critical for Universal Links to work.

The file must be served directly, without any HTTP redirects (including 301, 302, or 307). iOS will not follow redirects when attempting to fetch the AASA file, and even a redirect from http to https can cause the verification to fail.

Additionally, the response must be served with the correct Content-Type header:

Content-Type: application/json

The AASA file must be accessible over HTTPS, contain valid JSON, and be served as a raw file—not wrapped in HTML, compressed, or generated dynamically in a way that alters the response headers.

Because the requirements are strict and vary depending on your hosting provider or backend setup (e.g. Nginx, Apache, S3, Firebase Hosting, or a custom server), it’s easy to get this step wrong even if the file itself is correct.

In later posts, we’ll walk through concrete, step-by-step examples of hosting the AASA file correctly on different infrastructures and how to debug common issues when iOS fails to recognise your Universal Links.

Conclusion

Although Universal Links can seamlessly transition users from the web to your app, they only work when all the necessary components are correctly configured. Once your app and domain are properly associated, iOS can reliably determine whether to open your app or fall back to the browser when a link is clicked.

This post focuses on the core concepts needed to understand how Universal Links work. Follow-up posts will provide concrete setup examples and highlight common mistakes to avoid when implementing them.

The Page Object Model (POM) is the industry standard, but it is not easily scalable. The Flow Model Pattern improves upon the basic POM, offering a perfectly balanced solution that is not overly complex yet still scalable. The Screenplay Pattern is best suited for highly complex solutions and requires test engineers to have more technical knowledge. Therefore, before implementing it, ensure that your team can maintain a more abstract framework. If that’s not possible, either hold programming workshops or stick with the Flow Model Pattern.

Preface

Who should read this article?

• Test Automation Engineers, Technical Architects, and Software Developers seeking optimal end-to-end test automation solutions

• Anyone involved or interested in developing an end-to-end test automation strategy (e.g. Test managers, Test Automation Engineers, Technical Architects and Software Developers).

What is this article about?

After years of working with different test automation frameworks, I've learned that choosing the right design pattern isn't about finding the best one—it's about finding the right fit for your project and team.

The three primary patterns are:

• Page Object Model

• Flow Model Pattern (or Facade Design Pattern)

• Screenplay Pattern

This article explores when each pattern is most appropriate, drawing from my hands-on experience building and maintaining automation frameworks at different scales. I'll start with an overview of each approach, then guide you toward choosing the pattern that best fits your needs.

API-Based Model: A Universal Best Practice

Before exploring different design patterns, I want to highlight a principle that applies to all of them: the API-based approach to test automation.

My colleague Jovan Ilić advocates for this in his article "Test Automation: API-based Model", and I strongly recommend it regardless of which pattern you choose.

The Core Principle

Perform any action in the UI once, every other time use the API

Validate that the interface works, but use faster, more reliable API calls for test setup, navigation, and state management in subsequent tests.

Why It Matters

This approach delivers significant benefits with any pattern:

• Faster test execution

• Reduced flakiness

• Better test isolation

• Easier maintenance

• More focused UI coverage

Whether you use Page Object Model, Flow Model Pattern, or Screenplay Pattern, consider which interactions genuinely require UI validation and which can be handled more efficiently through APIs. This pragmatic approach will improve your test suite regardless of the architectural pattern you choose.

Page Object Model

Overview

The Page Object Model (POM) is the foundational design pattern that most test automation engineers learn first. The core idea is simple: separate page interaction logic from test scripts by encapsulating UI elements and their actions into dedicated page classes/files.

Description

The Page Object Model remains my preferred starting point for most projects—because of its proven effectiveness and universal recognition. This widespread adoption significantly shortens the ramp-up period for newcomers, as they can quickly adapt to a new codebase without learning an entirely new architectural approach.

While POM has its limitations (particularly as applications grow in complexity) its simplicity and directness make it an excellent foundation. Think of it as the baseline: straightforward to implement, easy to understand, and flexible enough to evolve as your project's needs change.

Code Example

// For the sake of simplicity and framework-agnosticity
// we're skipping the selectors/locators definitions and functions
// that are mentioned on the picture, but not used within test case

// ===========
// Test Script
// ===========
it('should add yellow t-shirt to cart', async () => {
    await ProductsPage.openProductDetails('tshirt-yellow-medium');
    await ProductDetailsPage.clickAddButton();

    const cartCount = await ProductDetailsPage.getCartCount();
    expect(cartCount).toBe(1);
});

// =============
// Products Page
// =============
export const ProductsPage = {
    async openProductDetails(id) { ... }
}

// ====================
// Product Details Page
// ====================
export const ProductDetailsPage = {
    async clickAddButton() { ... },
    async getCartCount() { ... }
}

Summary

✅ PROS

• Widely adopted standard – most common pattern in test automation

• Very intuitive – easy to understand for newcomers

• Strong community support – abundant tutorials, examples, and help available

• Flexible across all testing phases – DEV, SIT, UAT, etc.

• Reduces code duplication – reusable page classes across multiple tests

• Good separation of concerns – test logic separate from locators and interaction logic

❌ DOWNSIDES

• In pure form can get hard to maintain – repetitive code, page classes can become bloated (hundreds/thousands of lines)

• Violates SOLID principles – particularly Single Responsibility Principle

• Less efficient for applications where API testing would be more appropriate than UI testing

• UI-centric thinking – encourages testing everything through the UI

• Lack of standardisation – implementations vary widely between teams

• Poor scalability - becomes increasingly difficult to manage as application grows

Flow Model Pattern

Overview

Flow Model Pattern extends Page Object Model by adding a workflow abstraction layer between tests and page objects. This layer captures reusable business workflows, reducing code duplication and keeping both page objects and test scripts clean and focused.

Description

While developing my latest test automation framework, I found myself caught between two extremes: POMs felt too simplistic and led to duplication, while the Screenplay Pattern seemed too complex for a deadline-sensitive project where the team lacked experience with advanced patterns.

I needed something that combined the best of both worlds—as simple and straightforward as Page Object Model, but with better reusability. The solution was to add a single abstraction layer that would keep POMs focused on what they do best (mapping application pages) without the complexity of a full Screenplay implementation.

I called this approach User Steps, only to discover later while reading the ISTQB Test Automation Engineering Syllabus that it already had a name: Flow Model Pattern. Turns out I wasn't as original as I thought!

The last thing I want to mention about the Flow Model Pattern is that it isn't (very well) standardised, so you have complete flexibility to adapt it to your needs. This makes it particularly well-suited for small to medium teams who want better structure than pure POM without the learning curve of Screenplay Pattern.

Code Example

// For the sake of simplicity and framework-agnosticity
// we're skipping the selectors/locators definitions and functions
// that are mentioned on the picture, but not used within test case

// ===========
// TEST SCRIPT
// ===========
test('should add 20 t-shirts to cart', async () => {    
    await ShoppingFlow.addDiverseTShirts(20);

    const cartCount = await ShoppingFlow.getCartItemCount();
    expect(cartCount).toBe(20);
});

// =============
// SHOPPING FLOW
// =============
export const ShoppingFlow = {
    async addProductToCart(productId) {
        await ProductsPage.openProductDetails(productId);
        await ProductDetailsPage.clickAddButton();
        await ProductDetailsPage.goBackToProductsPage();
    },
    async addMultipleProducts(productIds) {
        for (const id of productIds) {
            await this.addProductToCart(id);
        }
    },
    async getCartItemCount() {
        return await ProductDetailsPage.getCartCount();
    },
    async addDiverseTShirts(count) {
        const tshirts = TShirtFactory.createDiverseSet(count);
        const productIds = tshirts.map(shirt => shirt.id);
        await this.addMultipleProducts(productIds);
    }
}

// ================================================================
// TEST DATA FACTORY - Clean way to handle the Test Data generation
// ================================================================
export const TShirtFactory = {
    create(color, size) {
        const id = `tshirt-${color.toLowerCase()}-${size.toLowerCase()}`;
        return {
            id: id,
            name: `T-Shirt Model ${color} ${size}`,
            color: color,
            size: size
        };
    },
    createDiverseSet(count) {
        const colors = ['yellow', 'black', 'blue', 'white', 'green', 'turquoise'];
        const sizes = ['small', 'medium', 'large'];

        return Array(count).fill(null).map((_, index) => {
            const color = colors[index % colors.length];
            const size = sizes[index % sizes.length];
            return this.create(color, size);
        });
    }
};

// =============
// PRODUCTS PAGE
// =============
export const ProductsPage = {
    async openProductDetails(id) { ... }
}

// ====================
// PRODUCT DETAILS PAGE
// ====================
export const ProductDetailsPage = {
    async clickAddButton() { ... },
    async getCartCount() { ... },
    async goBackToProductsPage() { ... }
}

✅ PROS

• Pragmatic middle ground – balances simplicity with scalability

• Reduces code duplication – workflows centralised in one place

• Keeps Page Objects clean – POMs focus on page interactions only

• Easy to learn – intuitive concept, quick onboarding

• Improves test readability – tests focus on WHAT, not HOW

• Quick to implement – can be added incrementally to existing frameworks

• No framework dependency – works with any tool

• Scales well for medium projects – handles growth without overwhelming complexity

❌ DOWNSIDES

• Not standardised – no official specification or industry consensus

• Boundary decisions – requires judgment on what goes in Flows vs Pages

• Limited documentation – far fewer resources than POM or Screenplay

• Can violate SOLID – risk of becoming a logic "dumping ground"

• Less structure – team must establish own conventions

• Risk of over-abstraction – can create unnecessary complexity if used too much

Screenplay Pattern

Overview

The Screenplay Pattern shifts from page-centric (or UI-centric) to actor-centric testing. Tests describe WHO performs WHAT actions to achieve their goals, using abilities, tasks, interactions, and questions. This actor-based approach excels in complex, multi-interface scenarios.

Description

Throughout my career working with all major test automation frameworks, I've observed an interesting pattern: while the Screenplay Pattern is well-known in the QA community, practical implementation experience remains relatively scarce. I've personally implemented it once in a production environment, and that experience—combined with extensive research—has given me valuable insights into why adoption remains limited.

This implementation gap isn't unique to my experience. The test automation community frequently grapples with understanding when and how to apply different patterns, as evidenced by ongoing discussions like the Reddit thread "I don't get the ScreenPlay Pattern". What's missing isn't just technical documentation—it's practical guidance on pattern selection based on real-world constraints and team capabilities.

This knowledge gap has significant implications. Many teams default to the Page Object Model and extend it with their best judgment—not because it's the optimal fit, but because it's familiar. Meanwhile, projects that could benefit from more sophisticated approaches continue using patterns that ultimately hinder their scalability and maintainability. My goal with this article is to bridge that gap by providing the contextual decision-making framework that's currently absent from most automation discussions—along with clear visual explanations that may finally help you "get the Screenplay Pattern”.

Why Screenplay Is Different

The Screenplay Pattern isn't just “POM with extra layers”—it's a fundamentally different way of thinking about test automation. Instead of organising code around pages and web elements, you organise around actors (users or systems) and their intentions.

Here's what makes it unique:

• Actor-centric thinking: Tests read like user stories ("James logs in and adds a product to cart")

• Separation of concerns: Abilities, Tasks, Interactions, and Questions each have single, clear responsibilities

• Composability: Small, reusable pieces combine to create complex workflows

• Multi-interface support: The same actor can interact with UI, API, and database seamlessly

• Scalability: Architecture that handles enterprise complexity without becoming unwieldy

The Trade-off

The price for this sophistication is complexity. Screenplay Pattern requires:

• Strong understanding of object-oriented design principles (especially SOLID)

• Longer initial setup and learning curve

• Team buy-in and training investment

• Commitment to the pattern's structure

For large, long-term projects with experienced teams, these investments pay dividends. For smaller projects or teams without strong programming backgrounds, the pattern can feel like over-engineering. To be completely honest, implementing it in such cases rarely makes sense – the return on investment simply won't justify the effort.

Code Example

// For the sake of simplicity and framework-agnosticity
// we're skipping the selectors/locators definitions and functions
// that are mentioned on the picture, but not used within test case

// ===========
// TEST SCRIPT
// ===========

// src/tests/Add20Tshirts.js
test('should add 20 t-shirts to cart', async () => {
    const sarah = Actor.named('Sarah');

    await sarah.attemptsTo(
        AddDiverseTShirtsToCart.count(20)
    );

    const cartCount = await sarah.asks(CartItemCount.value());
    expect(cartCount).toBe(20);
});

// ==========================================================================
// ACTOR – PEOPLE and EXTERNAL SYSTEMS interacting with the system under test
// ==========================================================================

// src/actor.js
export const Actor = {
    named(name) {
        return {
            name: name,
            ability: BrowseTheWeb,

            async attemptsTo(...tasks) {
                for (const task of tasks) {
                    await task.performAs(this);
                }
            },

            async asks(question) {
                return await question.answeredBy(this);
            }
        };
    }
};

// =====================================================================================
// ABILITIES – WRAPPERS around any INTEGRATION LIBRARIES (e.g. E2E, API, Cloud Services)
// =====================================================================================

// src/abilities.js
export const BrowseTheWeb = {
    async findElement(selector) { /* ... */ },
    async click(selector) { /* ... */ },
    async getText(selector) { /* ... */ }
};

// ==========================================================================
// TASKS - SEQUENCES OF ACTIVITIES as meaningful steps of a business workflow
// ==========================================================================

// src/tasks/AddDiverseTShirtsToCart.js
export const AddDiverseTShirtsToCart = {
    count(numberOfShirts) {
        return {
            numberOfShirts: numberOfShirts,

            async performAs(actor) {
                const tshirts = TShirtFactory.createDiverseSet(this.numberOfShirts);

                for (const tshirt of tshirts) {
                    await actor.attemptsTo(
                        AddProductToCart.withId(tshirt.id)
                    );
                }
            }
        };
    }
};

// src/tasks/AddProductToCart.js
export const AddProductToCart = {
    withId(productId) {
        return {
            productId: productId,

            async performAs(actor) {
                await actor.attemptsTo(
                    OpenProductDetails.for(this.productId),
                    Click.on('.add-button'),
                    Click.on('.back-to-products')
                );
            }
        };
    }
};

// ================================================================================
// INTERACTIONS - LOW-LEVEL ACTIVITIES an ACTOR can perform using a given interface
// ================================================================================

// src/interactions/OpenProductDetails.js
export const OpenProductDetails = {
    for(productId) {
        return {
            productId: productId,

            async performAs(actor) {
                const selector = `[data-product-id="${this.productId}"]`;
                await actor.attemptsTo(
                    Click.on(selector)
                );
            }
        };
    }
};

// src/interactions/Click.js
export const Click = {
    on(selector) {
        return {
            selector: selector,

            async performAs(actor) {
                await actor.ability.click(this.selector);
            }
        };
    }
};

// ====================================================================================
// QUESTIONS - RETRIEVE INFORMATION from the system under test and the test environment
// ====================================================================================

// src/questions/CartItemCount.js
export const CartItemCount = {
    value() {
        return {
            async answeredBy(actor) {
                const cartBadge = await actor.ability.findElement('.cart-badge');
                const text = await actor.ability.getText('.cart-badge');
                return parseInt(text, 10);
            }
        };
    }
};

// ================================================================
// TEST DATA FACTORY - Clean way to handle the Test Data generation
// ================================================================

// src/utils/TShirtFactory.js
export const TShirtFactory = {
    create(color, size) {
        const id = `tshirt-${color.toLowerCase()}-${size.toLowerCase()}`;
        return {
            id: id,
            name: `T-Shirt Model ${color} ${size}`,
            color: color,
            size: size
        };
    },

    createDiverseSet(count) {
        const colors = ['yellow', 'black', 'blue', 'white', 'green', 'turquoise'];
        const sizes = ['small', 'medium', 'large'];

        return Array(count).fill(null).map((_, index) => {
            const color = colors[index % colors.length];
            const size = sizes[index % sizes.length];
            return this.create(color, size);
        });
    }
};

Summary

✅ PROS

• Highly maintainable

• Multi-interface support – seamlessly combines UI, API, and database interactions

• SOLID design principles – follows Single Responsibility, Open-Closed principles

• Business-focused language – tests read like user stories/workflows

• Better abstraction layers – Tasks, Actions, Questions provide clear organisation

• Excellent scalability – handles complex, multi-actor scenarios well

• Reusable components – interactions and tasks can be composed and reused

❌ DOWNSIDES

• Steeper learning curve – much more complex concepts to understand initially

• Limited community knowledge – fewer practitioners with hands-on experience

• Requires stronger technical skills – team needs understanding of design patterns

• Overkill for simple projects – unnecessary overhead for small and even medium-sized applications

• Framework dependency – typically requires specific frameworks (e.g. Serenity/JS, Boa Constrictor, ScreenPy)

• Longer onboarding time – new team members take longer to become productive

Choosing the Right Approach

Here are my recommendations for different project settings. As a general rule of thumb: stick with the Keep It Stupidly Simple (KISS) paradigm and don't over-engineer in the early stages.

When it comes to different project contexts, here are my suggestions:

If you have any questions, feel free to leave a comment.

Thanks and happy coding!

In a previous article from the start of this year, I mentioned various frameworks that can be used for mobile test automation. The one that came up repeatedly was WebdriverIO, which I have been using actively for over two years now.

It is a shame that there are no really comprehensive articles on how to use WebdriverIO, especially when it comes to automating complex scenarios such as cross-platform testing of iOS, Android and Web at the same time.

Thanks to Cloudflight's courtesy, I was able to open-source the WebdriverIO sandbox that I created for internal use and knowledge sharing. I will use this as the basis for this article.

https://github.com/cloudflightio/cross-platform-test-framework

The demo setup of the sandbox framework is based on the well-known Wikipedia application.

Getting Started

Want to try out the framework?

The complete setup guide with all prerequisites, installation steps, and your first test run is available in the project README.

Quick start for the impatient

yarn install
yarn run wdio:web:edge

The framework supports Web (Edge), Android, and iOS testing out of the box. Detailed platform-specific setup instructions are in the README.

Deep dive

Why yet another Boilerplate Project?

Before we take a closer look at the features of our sandbox, I would like to briefly explain why I felt the need to create it in the first place.

When you start working with the WebdriverIO, the Boilerplate Projects page is one of the first places you’ll visit. Although there are a plethora of different setups, I couldn’t find anything that suited my personal needs.

You might be wondering what those needs were. Here are the features that I couldn't find in any of the available boilerplate projects:

1. Standardised, simple selectors handling — consistent patterns that work across all platforms.

2. The Page Object Model pattern — an industry standard with which every test automation engineer should be familiar. It's effective because it's simple and intuitive.

3. Simple, maintainable architecture — avoiding over-engineering and façade frameworks (like Cucumber) that add disproportionate complexity without delivering equivalent practical value. In my 6+ years of test automation experience, over-engineered frameworks consistently become maintenance burdens when the original developers leave, often requiring teams to reverse engineer their own codebase.

4. Write-once-run-everywhere test spec files — we'll explore this concept in detail later in this article.

5. Simple reporting integrated directly into the boilerplate — no additional setup required.

6. A true cross-platform example implementation covering all major platforms (Web, Android, and iOS) that works out-of-the-box.

Now that we've outlined these pain points, let's examine the most important ones and see how our framework solves them.

Selectors Handling

The select() Function Overview

The word simple that I'm using here might seem subjective at first glance. Especially, if you take a look at the implementation of the code, where we have the following:

// test/common/sharedCommands.ts
export function select(selector: Selector): ChainablePromiseElement {}
export function selectArray(selector: SelectorArray): ChainablePromiseArray {}

// test/common/selectors.ts
export const selectors = {
    homePage: {
        searchField: {
            android: 'org.wikipedia:id/search_container',
            ios: '**/XCUIElementTypeSearchField[`label == "Search Wikipedia"`]',
            web: '(//input[@name="search"])[1]',
            mobileBrowser: '//form[@id="minerva-overlay-search"]//input[@name="search"]',
        },
    }
}

// test/pageobjects/home.page.ts
searchField: () => select({
    ...selectors.homePage.searchField,
    iosSelectionMethod: getByClassChain,
}),

It looks anything but simple.

However, when you consider the alternatives—how would you approach this differently?—you can see why this solution is actually elegant and robust.

To illustrate this, here's how you might approach cross-platform selectors in a naive way:

// *************
// THE NAIVE WAY
// *************

// test/pageobjects/home.page.ts - example without selectors handling
searchField: async () => {
    if (browser.isNativeContext) {
        return browser.isAndroid ?
            await browser.$('(//android.widget.TextView[@text="Appium"])[1]') :
            await browser.$('-ios class chain:**/XCUIElementTypeStaticText[`label == "Appium"`][2]');
    } else {
        return browser.isMobile ?
            await browser.$('//input[@name="search"])[1]') :
            await browser.$('//form[@id="minerva-overlay-search"]//input[@name="search"]');
    }
}

// ********************
// THE STANDARDISED WAY
// ********************

// test/pageobjects/home.page.ts
searchField: () => select({
    ...selectors.homePage.searchField,
    iosSelectionMethod: getByClassChain,
}),

// test/common/selectors.ts
export const selectors = {
    homePage: {
        searchField: {
            android: 'org.wikipedia:id/search_container',
            ios: '**/XCUIElementTypeSearchField[`label == "Search Wikipedia"`]',
            web: '(//input[@name="search"])[1]',
            mobileBrowser: '//form[@id="minerva-overlay-search"]//input[@name="search"]',
        },
    }
}

With just one selector, this might not seem like much of an improvement, but add a few more selectors and the naive approach becomes ridiculous compared to our standardised way:

// *************
// THE NAIVE WAY
// *************

// test/pageobjects/home.page.ts - example without selectors handling
searchField: async () => {
    if (browser.isNativeContext) {
        return browser.isAndroid ?
            await browser.$('org.wikipedia:id/search_container') :
            await browser.$('-ios class chain:**/XCUIElementTypeSearchField[`label == "Search Wikipedia"`]`][2]');
    } else {
        return browser.isMobile ?
            await browser.$('//input[@name="search"])[1]') :
            await browser.$('//form[@id="minerva-overlay-search"]//input[@name="search"]');
    }
},
searchResultItem: async () => {
    if (browser.isNativeContext) {
        return browser.isAndroid ?
            await browser.$('id:org.wikipedia:id/page_list_item_title') :
            await browser.$('-ios predicate string:label == "Appium"');
    } else {
        return browser.isMobile ?
            await browser.$('//li[@title="Appium"]') :
            await browser.$('//li[@title="Appium"]//a');
    }
}

// ********************
// THE STANDARDISED WAY
// ********************

// test/pageobjects/home.page.ts - an actual excerpt from the codebase
searchField: () => select({
    ...selectors.homePage.searchField,
    androidSelectionMethod: getById,
    iosSelectionMethod: getByClassChain,
}),
searchResultItem: () => select({
    ...selectors.homePage.searchResultItem,
    androidSelectionMethod: getById,
    iosSelectionMethod: getByPredicateString,
}),

And don't even get me started on handling the Android and iOS selection methods—just look at the id:, -ios class chain:, and -ios predicate string: prefixes in the naive approach.

The selectArray() Function Overview

Since we thoroughly covered select(), I don't think we need to delve deeply into selectArray() since it functions similarly. However, it's important to explain why we need it.

The best time to use the selectArray() function is generally when you need to count the number of elements.

Some might argue that it's useful for extracting an array of WebElements. That's a fair point, but I strongly recommend getting familiar with using variables in selectors (covered in the next section). For me, the select() function combined with variables is sufficient 99% of the time.

One more thing: select() and selectArray() are simply facades over regular $ and $$ WebdriverIO’s commands, and since they return the ChainablePromiseElement and ChainablePromiseArray, you can refer to the official WebdriverIO documentation for more information on how to use them in more complex cases:

1. https://webdriver.io/docs/selectors

2. https://webdriver.io/docs/api/browser/$

3. https://webdriver.io/docs/api/browser/$$

Selectors with Variables

Sometimes, we want to use selectors with variables. The most common scenario would be an attribute that has a dynamically generated counter (e.g., item-0-dropdown, item-1-dropdown etc.).

Without Variables

Let's start again with the naive, straightforward approach—we might end up with a hardcoded selector like this.

// *************
// THE NAIVE WAY
// *************

// test/common/selectors.ts - example without variables
homePage: {
    firstDropdownItem: {
        ios: '//XCUIElementTypeStaticText[@name, "item-0-dropdown"]',
        android: '//android.widget.TextView[@text, "item-0-dropdown"]',
        web: '//*[@data-testid="item-0-dropdown"]',
        mobileBrowser: '//*[@data-testid="item-0-dropdown"]//span',
    },
    secondDropdownItem: {
        ios: '//XCUIElementTypeStaticText[@name, "item-1-dropdown"]',
        android: '//android.widget.TextView[@text, "item-1-dropdown"]',
        web: '//*[@data-testid="item-1-dropdown"]',
        mobileBrowser: '//*[@data-testid="item-1-dropdown"]//span',
    },
    thirdDropdownItem: {
        ios: '//XCUIElementTypeStaticText[@name, "item-2-dropdown"]',
        android: '//android.widget.TextView[@text, "item-2-dropdown"]',
        web: '//*[@data-testid="item-2-dropdown"]',
        mobileBrowser: '//*[@data-testid="item-2-dropdown"]//span',
    }
}

// test/pageobjects/home.page.ts - example without variables
firstDropdownItem: () => select(selectors.homePage.firstDropdownItem)
secondDropdownItem: () => select(selectors.homePage.secondDropdownItem)
thirdDropdownItem: () => select(selectors.homePage.thirdDropdownItem)

// test/pageobjects/home.page.ts - example code usage
async selectDropdownItemByIndex(itemIndex: number): Promise<void> {
  switch (itemIndex) {
    case 0:
      await this.firstDropdownItem().click();
      break;
    case 1:
      await this.secondDropdownItem().click();
      break;
    case 2:
      await this.thirdDropdownItem().click();
      break;
  default:
    throw new Error(`Invalid dropdown item index: ${itemIndex}.`);
  }
},

With Variables

The second method allows us to provide dynamic data, and make our tests more robust.

To use variables, create a selector providing the variable in curly braces {{itemIndex}}, then pass the variable to the select() or selectArray() function within your Page Object Model file.

// ********************
// THE STANDARDISED WAY
// ********************

// test/common/selectors.ts - example with variables
homePage: {
    dropdownItem: {
        ios: '//XCUIElementTypeStaticText[@name, "item-{{itemIndex}}-dropdown"]',
        android: '//android.widget.TextView[@text, "item-{{itemIndex}}-dropdown"]', 
        web: '//*[@data-testid="item-{{itemIndex}}-dropdown"]'
    }
}

// test/pageobjects/home.page.ts - example with variables
nthDropdownItem: (dropdownItemIndex: number) =>
    select({
        ...selectors.homePage.dropdownItem,
        variables: { itemIndex: `${dropdownItemIndex}` },
    }),

// test/pageobjects/home.page.ts - example code usage
async selectDropdownItemByIndex(itemIndex: number): Promise<void> {
  await this.nthDropdownItem(itemIndex).click();
},

Simple & Maintainable Architecture

End-to-end tests are not regular applications, period.

While abstractions, inheritance, SOLID, DRY, and design patterns have their place in test automation, you must adjust your mindset. In most cases, you'll be working with less experienced developers or testers fresh from programming courses who lack battle-tested experience. Over-engineering your test framework with advanced patterns may seem elegant, but it often creates a maintenance nightmare for the very people who need to work with it daily.

If you reach the point where you feel your test scripts need their own tests to keep everything in place—you're doing something wrong.

Overly complex frameworks only work when you have senior test automation engineers on the team—but from an economic perspective, most companies can't justify hiring multiple senior-level testers. Moreover, since senior test automation engineers possess skills comparable to senior software developers, they often transition into software development roles for better career opportunities unless they have a genuine passion for testing.

The solution? Focus on the KISS principle

Keep It Stupidly Simple (KISS)—write code that anyone can work with.

The ideal scenario: your test scripts should be simple enough that new tests can be created through straightforward copy-paste with minimal modifications.

Build from the bottom-up: write the simplest code that works first, then refactor. That's why our framework uses straightforward building blocks:

• Page Object Models — encapsulate page interactions and behaviors

• Centralized Selectors — single source of truth for element identification

• Test Scripts — contain your test logic and assertions

• Configuration Files — pre-configured settings that let you start testing immediately

• Shared Commands — utility functions you can freely modify depending on your needs, avoiding dependencies on inflexible third-party libraries that require forking to extend

• Custom Matchers — extend default WebdriverIO assertions with custom message logging for better reporting and easier debugging

• Flows (optional, for larger frameworks) — reusable test sequences (learn more)

Write-Once-Run-Everywhere Test Spec Files

When I first encountered cross-platform automation, I was completely clueless about how it should work. But as I started writing my first test cases, I quickly realized there was potential for standardization.

If your application looks and behaves the same across all platforms, it's super easy to automate using our framework. The main effort is extracting selectors for each platform (DevTools for web, Appium Inspector for mobile). If your application behaves slightly differently but shares most functionality, simple conditional statements handle the variations.

The best part? You can reuse the same test code across all platforms, thanks to WebdriverIO's extreme versatility.

Below is an excerpt from our Wikipedia test example, demonstrating how platform-specific variations are handled while keeping the script readable and contained in a single file:

describe(`Wikipedia`, () => {
  it('Search for an article about "Test"', async () => {
    await addTestId('TEST-1');

    if (!browser.isNativeContext) {
      await homePage.openUrl();
    }

    if (browser.isNativeContext) {
      await homePage.pressSkipButton();
    }

    await homePage.enterTextToSearchField('Test');
    await homePage.pressFirstSearchResultItem();
    await wikipediaGamesModal.closeModal();
    await articlePage.waitForPageLoad();
    const pageTitle = await articlePage.getPageTitle();

    await expect(pageTitle).toEqualString('Test');

    await takeScreenshotWithTitle('Successful test - Page title with the "Test" value');
  })
})

Simple Reporting Integrated Directly into the Boilerplate

I don't know about you, but I really dislike having too many choices when setting up a new technology. I want something decided for me, with the flexibility to change it if needed.

The reason? A shorter ramp-up period. When you have a clear picture of what's what, you can learn it quickly and then fine-tune. Simply put—I just want to drive a car, not assemble it from a DIY kit.

That's why our framework has reporting baked in out of the box, with three layers:

1. Basic logs — WebdriverIO's standard console output

2. Extended log steps — Custom logs embedded in shared commands that output test steps descriptions to both the console and Allure reports

3. Allure Report — Post-test HTML report that can be launched locally with allure serve or hosted (e.g., on GitHub Pages)

Wrapping Up

If you made it this far, thank you for your time. I hope you now have a clearer understanding of what our Test Automation Framework can do for you and how it can speed up your WebdriverIO-based cross-platform end-to-end setup.

If you have any questions, feel free to leave a comment. Contributions are welcome—feel free to open pull requests. While we don't have a formal code of conduct for contributors yet, that's something I plan to add in the future.

Thanks and happy coding!

Now, imagine this is happening in your company’s payroll and budgeting software. A single misleading label or poorly worded error message can have serious consequences, from lost data to financial impacts. UX writing in B2B isn’t just about making things sound nice - it’s about making sure users can do their jobs. It’s the difference between “No data found” and “No entries for this period. Try adjusting your filters.” It’s also about knowing when to keep it formal, when to add a touch of personality, and when to just stay out of the way.

So, let’s dive into the world of UX writing for B2B SaaS - the place where clarity meets complexity, tooltips become lifesavers, and error messages don’t make you want to throw your laptop out the window.

So, what is UX writing?

UX writing is the art of writing clear, concise and functional text that guides users through digital products. It’s the microcopy you see on buttons, error messages, empty states, onboarding instructions and tooltips—everywhere text helps users take action.

While the fundamentals of usability remain the same, the design approach in enterprise products tends to differ from consumer. Enterprise products are more often than not complex workflow tools that involve organizing and displaying large amounts of data. This makes strong, well-structured design essential to support users getting their jobs done.

Keep it simple? Not always

Simplicity is a core design principle, but in B2B software oversimplifying a feature through words can undermine its meaning or the consequence of an action. Workflows in enterprises are complex, and the tools must work in that complexity. What does this mean in terms of UX writing?

Use terms and language your audience understands

Enterprise users are people with specific industry expertise, and they expect to see the terms and phrases relevant to their work. Removing or oversimplifying those terms in an attempt to "keep it simple" can create unnecessary friction and even undermine a feature’s meaning or consequence of an action. UX writers must familiarize themselves with the language their users already understand. One of the best ways to do this is by collecting domain knowledge from users, stakeholders, product managers and other sources like internal training documents or customer support tickets.

Abbreviations matter

Abbreviations are another challenge in B2B UX writing. While they are often necessary in technical or industry-specific contexts, they must be used consistently and, when possible, explained in a way that supports both new and experienced users. For example, an abbreviation might be well known within a particular industry but unfamiliar to a new hire or someone switching from a competitor’s product. Striking the right balance between precision and comprehensibility is key.

Product guidance: Hello tooltips!

B2B SaaS products often come with a learning curve, especially when they involve complex user flows and multiple user roles. Most onboarding information is only needed as a one-time learning and well-placed tooltips can provide just-in-time guidance without overwhelming the user.

These micro-interactions should be brief and helpful, offering just enough information to guide the user to the next step. A tooltip that appears when hovering over a setting, for example, can explain what the option does without requiring the user to leave their workflow to search for documentation.

Delight with caution: Striking the Right Balance in Enterprise UX

While consumer products often add delight through playful copy and animations, this approach doesn’t always translate well to enterprise software. Users handle complex tasks and delight can become a distraction. Many users are already experts in the tool and rely on it daily, often repeating the same workflows. Flashy animations or quirky copy can quickly feel annoying and disruptive.

This doesn’t mean there’s no room for delight—it just needs to be subtle and well-timed. By being very thoughtful and finding innovative ways to add value, you can create micro-experiences that make someone smile or inspire them.

Here’s a couple of ideas that can make a product feel more engaging:

• Language: Use positive, active language and balance your brand voice with the language users actually speak.

• Empty states: Have smart empty states or defaults, teach users something they didn’t know.

• Success Messages: Find small moments of celebrations to share.

Final thoughts

UX writing in B2B is about more than just making things sound good—it’s about clarity, precision, and helping users get their work done. The goal is to use the right language for the right audience. Whether it's through well-placed tooltips, clear error messages, or actionable labels, good UX writing reduces friction, improves efficiency, and ultimately leads to higher product adoption and satisfaction.

For teams working on B2B products, now is the time to take a closer look at the microcopy within your application. Are labels explicit? Are tooltips helpful? Are error messages guiding users rather than frustrating them? Small changes in UX writing can have a big impact, making the difference between a product that feels intuitive and one that users struggle with.

What will this series be about?

During this series of blog posts, I will do my best to provide my opinion on specific models, determine which use case each one fits, discuss the pros and cons of using that model, and more. I’d like to start with some general notions and first steps, and then deep-dive into different models and particularities of those models. Disclaimer: All opinions on usage and prompts are from personal experience with these models.

What prompted this?

Year 2025 has been a year for new beginnings in my life. Among these are new personal projects. And the frustrations that sometimes come with them. During a rant to a friend about such frustrations (looking at you, configuration issues), he asked me if this is the tipping stone and if I will finally give ChatGPT a chance.

Now, when it comes to technology, especially software, I am one of those people who want to try the new shiny thing as soon as possible and test the limits. However, there is one piece of tech I avoided since it came out: LLMs. I simply did not believe that technology was there. However, I was proven at least partially wrong when I finally tried it for my personal project.

Data privacy and mistakes, or why I avoided LLMs for so long

One of the main issues I had with LLMs is that their data privacy rules were not super clear in the beginning. In an ideal world, such information would be only used to provide the answer and, for longer conversations, to be saved locally on the device and referenced only in the context of that conversation.

However, that has somewhat improved in recent times. For example, OpenAI, Microsoft, and Anthropic do not use the prompts and generated answers for training, while others, like Google, gate this control behind a paywall. Or, in the case of DeepSeek, don’t even mention an opt-out feature.

If we look at how the data is stored, we have the following cases:

• OpenAI - 30 days

• Microsoft - configurable or ZDR (Zero Data Retention)

• Google - 3 years if randomly selected for human review. 3 days with activity tracking off

• Anthropic - unspecified

• DeepSeek - unclear

Then, there is the issue of ownership over the prompts and results. OpenAI, Anthropic, and Microsoft specify that the user has ownership, while Google imposes some restrictions (the data of free-tier users can be accessed by others and is used in training). DeepSeek is again vague, since it uses user data by default for improving the model. There is one thing all the models used as examples above agree on: no copyright guarantee.

The second issue that plagues LLMs is hallucinations and the mistakes resulting from that (how many rocks did I need to eat? One small rock per day?). So, never take what an LLM says for granted.

What is better? Online hosted or self-hosted?

While there are more ways to use an LLM other than Online and Self-hosting, these are probably the most accessible to the average user. So I will focus on these methods at the moment.

I will start with what I would call online hosting. This is the method in which you use the LLM the way it is provided by the creator company, using their apps or websites with an account created and with the settings tuned to your needs.

This is the easiest way, and it is great if you need speed, want to have state-of-the-art outputs, and zero overhead. However, there are drawbacks. You can’t guarantee privacy, you will need permanent network access, and you are always going to worry about your token limit. And sometimes things might just not work, and you will need to wait for some internal service to be back online.

Self-hosting, on the other hand, fixes some of these issues: in terms of privacy and network connection, you are in control. And in some cases, you can even tune the weights. But the machine or machines these run on need to have some competent hardware, the setup time might not be as fast, and it is unlikely that the self-hosted model is state-of-the-art.

So, in conclusion, every person who wants to start using LLMs in their daily life or work will have to decide the best way of accessing them. For me and my project, self-hosting will be the way forward.

What’s next?

Next time, we’ll talk about the first model that I had some more serious contact with: GPT-3.5. I know this is not an open model, but we will use this as a comparison base for the rest. And, additionally, we will discuss the first steps of self-hosting.

A New Foundation for Workflow Orchestration

The biggest additions of Airflow 3.0 at a glance:

• Event-based triggers

• Workflow (DAG) versioning

• New react-based UI

• Asset notation

• Backfills

One of the most anticipated additions to Airflow 3.0 is its support for event-driven scheduling. Previously, Airflow’s strength was in time-based, cron-style orchestration. While this model works well for batch pipelines, it struggles in scenarios where real-time responsiveness is critical. With 3.0, workflows can now respond to data events, such as files appearing in cloud storage buckets or updates occurring in databases, enabling near-real-time orchestration. This positions Airflow to handle streaming and micro-batch use cases more elegantly than ever before (DataCamp).

Another major advancement is built-in DAG versioning, which allows every execution of a Directed Acyclic Graph (DAG) to be tied to a specific, immutable snapshot of its definition. This feature significantly improves debugging, traceability, and auditing, particularly for organizations in regulated industries where compliance and reproducibility are crucial. The versioning helps answer the critical question of what code is executed at which point in time.

Airflow 3.0 also comes with a completely overhauled web UI, rebuilt using modern frontend technologies. The new interface delivers faster performance and a cleaner user experience. With improved tools for visualizing DAG runs, managing tasks, and inspecting logs, the user interface is no longer a pain point, but a productivity enhancer.

Furthermore, the new asset-centric syntax enables developers to use the @asset decorator to define workflows directly around data assets. This reduces boilerplate and aligns pipeline logic more naturally with the data itself. In practice, it shifts the paradigm from orchestrating tasks to orchestrating data—a subtle but meaningful conceptual leap.

Lastly, scheduler-managed backfills eliminate the hassle of managing historical data reprocessing through fragile CLI commands. Backfills can now be triggered, paused, and monitored directly from the UI or API, dramatically simplifying historical data correction.

Apart from newly added features, Airflow 3.0 also brings some improvements under the hood. Astronomer.io, a key driving force behind Airflow’s ongoing development, played a crucial role in the rollout of Airflow 3.0. As detailed in their overview, the most substantial optimizations include:

• A Faster Scheduler: The Airflow 3.0 scheduler is optimized for speed and scalability, reducing latency during DAG processing and enabling faster task execution feedback.

• Active Dependency Management: Improved dependency tracking increases responsiveness and execution efficiency, particularly for complex pipelines.

• Database Connectivity Improvements: Airflow now interacts with metadata databases more efficiently, improving stability and reducing load on the backend.

• Upgrade Path Enhancements: Astronomer has simplified the upgrade path for both self-hosted and managed Airflow environments, making it easier for organizations to move from previous versions to 3.0.

Advantages That Elevate the Experience

The improvements in Airflow 3.0 offer several benefits. Not only the changes in the architecture, but also the updated UI enhance observability and performance, giving engineers better insight into the health of their workflows through logs, metrics, and task statuses.

Airflow’s maturity continues to be one of its biggest strengths. With wide adoption across industries and robust community support, organizations can confidently deploy and scale their pipelines using a platform with proven reliability. Integration with other tools, like dbt (data build tool), remains strong, allowing users to orchestrate transformations seamlessly in concert with data ingestion and extraction workflows (see: Execute DBT with Airflow and Cloud Run).

Pain Points That Still Remain

Despite the significant improvements introduced in Airflow 3.0, the platform still carries several challenges that may hinder its adoption and use. One major issue is its steep learning curve, which remains a roadblock for many teams. Configuring DAGs, setting up deployment infrastructure, and troubleshooting failures often require deep technical expertise and careful coordination between multiple components.

Running Airflow at scale continues to be a resource-intensive undertaking, particularly for organizations managing hundreds or thousands of orchestrated tasks per day. Additionally, while Airflow 3.0 has taken strides forward, it still doesn’t fully embrace streaming workflows yet.

Historically, Airflow has faced criticisms related to scheduler instability, DAG deadlocks, and challenges with local testing—problems often discussed in forums like this Reddit thread. Although some of these issues have been alleviated in version 3.0, others persist and remain in areas that require further focus and improvement.

Challenges Specific to Airflow 3.0

While Airflow 3.0 introduces several new features, many are underdeveloped and fall short of being ready for seamless use. For instance, the implementation of Asset adapters and Event-based triggers appears incomplete. These much-anticipated features lack polish for seamless developer experience, by only supporting a limited amount of connectors right out of the box. Similarly, the UI for visualizing Assets and DAGs can be confusing, leaving users struggling to fully understand or manage the relationships between these components.

Another pain point relates to DAG versioning, which is unable to effectively track changes to dbt models using cosmos - a capability that many may find invaluable.

The overall developer experience in Airflow 3.0 also continues to feel clunky. From initial setup to debugging, developers starting out with Airflow may struggle with the platform’s tools and processes. This is coupled with the complex nature of deployments, which are especially challenging in advanced scenarios, such as setting up Airflow on Kubernetes or managing hybrid-cloud environments.

One of the most noticeable drawbacks of Airflow 3.0 is the insufficient or missing documentation for its new features. Teams are left without clear guidance on several critical areas, such as:

• How to connect Assets with classic DAGs.

• How to implement and use Asset adapters.

• How to configure and utilize Event-based triggers effectively.

This lack of comprehensive documentation creates a barrier for teams trying to explore and adopt the new functionalities introduced in Airflow 3.0. Without practical examples or detailed tutorials, understanding the usage of these features from the source code can be cumbersome.

Comparing Airflow 3.0 to Other Orchestration Tools

In the rapidly evolving world of orchestration tools, how does Airflow 3.0 stack up?

Prefect has long marketed itself as a more Pythonic and lightweight alternative to Airflow. It offers seamless local development, easy debugging, and a simple function-based interface. For teams seeking quick setup and modern developer experience, Prefect is hard to beat. However, Prefect may fall short in highly complex enterprise environments where Airflow’s fine-grained scheduling and extensibility are still unmatched. In comparison, Prefect is easier to use, Airflow’s integrations and broader community support make it more suitable for complex, regulated workflows.

Dagster, by contrast, places a heavy emphasis on data assets and observability. It provides a more modular, testable, and development-friendly interface for building pipelines. Dagster’s partitioning mechanism, better support for local development, and clear delineation between production and staging make it a favorite among data teams with engineering-heavy workflows. Airflow 3.0 narrows this gap considerably with its asset-centric features and improved developer tooling, but Dagster still feels more modern in its architecture. Still, the smaller community and the missing RBAC support for the open source version might be a deal breaker for some projects.

Other tools like Kestra, Shipyard, and DataChannel each have their own niches, often targeting ease-of-use or native SaaS integrations, but they don’t yet match Airflow’s flexibility. Azure Data Factory (ADF) offers strong native integration with the Azure ecosystem and is ideal for Microsoft-heavy shops, though it lacks the open-source extensibility that defines Airflow.

Final Thoughts

Apache Airflow 3.0 is a big release that reaffirms the platform’s relevance and adaptability. It addresses many of the performance, usability, and reliability issues that have long plagued the system, while introducing forward-looking features that make it competitive with modern orchestrators like Prefect and Dagster.

While the platform still demands operational expertise and thoughtful architecture to run effectively, it now offers a significantly better out-of-the-box experience. For organizations that need to scale complex workflows with a high degree of control and visibility, Airflow 3.0 offers one of the most mature and capable orchestration solutions on the market today.

As mentioned, there are areas that still need attention from the Airflow community. The immaturity of new features, the incomplete documentation, and the ongoing challenges with deployment complexity and usability can create frustration for teams. For organizations already invested in Airflow, these might be manageable issues, but for those evaluating orchestration tools, they could serve as deterrents. In some cases, tools like Dagster or Prefect, with their more intuitive workflows and seamless integrations, may still be more attractive alternatives. Airflow 3.0 has set a strong foundation for future growth, but there is still work to be done to make it a truly developer-friendly and scalable solution.

While I have no doubt that the new Angular Material is great, there is one aspect that might surprise you later. In this article, I would like to share with you my experience.

Is it easy to update to Material 19 from the previous version?

No, not at all. But we got used to it recently, didn't we? Joking aside, there is a guide and also a migration script you can perform via ng update, but definitely some manual intervention will be needed.

Angular Material 19 - What is so different?

In the new version, it is the theming that makes the difference. Theming is what "makes you customize colors and typography"[1] in your app, and while Angular Material 17 works with M2 (Material Design 2), Angular Material 18 and 19 work with M3 (Material Design 3).

I really think the new M3 works pretty well, but to help people understand the idea behind it, there are often nice parts in the documentation that compare M3 to M2 when explaining things like color roles or elevation effects.

Past and Present - practical examples

How we used to define themes in Angular Material 17:

@use '@angular/material' as mat;

$my-primary: mat.define-palette(mat.$indigo-palette, 500);
$my-accent: mat.define-palette(mat.$pink-palette, A200, A100, A400);

$theme: mat.define-light-theme((
 color: (
   primary: $my-primary,
   accent: $my-accent,
 ),
 // typography + density
));

How we used to define themes in Angular Material 18:

@use '@angular/material' as mat;

$theme: mat.define-theme((
  color: (
    theme-type: dark,
    primary: mat.$violet-palette,
  ),
  // typography + density
));

How it is done now in Angular Material 19:

@use '@angular/material' as mat;

html {
  color-scheme: light dark; // can be light, dark, or both (like here)
  @include mat.theme((
    color: mat.$violet-palette,
    // typography + density
  ));
}

It may not look that different at first glance, but there are 2 key things:

• in v17 and v18 we store the created theme in the SASS variable $theme, we do not do that in v19

• v19 uses color-scheme, whereas in v17 and v18 the theme itself is either light or dark

SASS variable $theme is not needed anymore

Prior to v19, referencing themes and colors was not ideal. Anytime we needed to get a color from a theme and use it, we had to find/import the created theme + also import material tools, and then use them to extract some color:

@use '@angular/material' as mat;
@use 'theme' as theme;

.color-primary {
  color: mat.get-color-from-palette(map-get($theme, primary));
}

What happens inside the v19 theme definition is the creation of globally accesible properties. So the same code looks like this:

.color-primary {
  color: var(--mat-sys-primary);
}

This is much nicer as we do not need a huge amount of references and @import/@use/@forward rules just to get some color. You can read about all possible color roles in the M3 documentation.

CSS color-scheme vs explicit light/dark

color-scheme is a CSS property that allows html elements to indicate which color scheme to use - if dark or light

Although this is nothing new, I often find this feature missing, struggling with dark pages during the day, or bright screens at night. In my personal opinion this can give the user a much better experience, and here we have all the tools to do it, so please:

Never hardcode colors!!!

Now we all know that we can easily create our pages to support light/dark themes automatically, but then why is there “Oh NO“ in the title?

Yes, everything is fine with this color-scheme, the reason is a CSS function called light-dark on which all this functionality is based on.

light-dark

light-dark is a CSS function that simply selects 1 color out of 2 based on - yes, you are right - based on color-scheme. So if you inspect your browser, you will see, that var(—mat-sys-primary) (which we used to set the color) does not hold #005cbb, but it holds light-dark(#005cbb, #abc7ff).

All is well unless you check the compatibility table for the light-dark function. I did not do that. Until someone opened my page on a Macbook and reported bug.

Long story short - light-dark on Apple devices is only available from 17.5, so the probability that someone e.g. does not have the latest iOS is high.

It was a nice try, but should we revert now?

No, there is still a way!

We can still combine the old way of defining themes and reading scheme preferences with the new Angular Material 19. My solution to this problem was the following:

html {
  @include mat.theme((
    color: (
      primary: mat.$violet-palette,
      theme-type: light,
    ),
    // typography and density
  ));
}

@media (prefers-color-scheme: dark) {
  html {
    @include mat.theme((
      color: (
        primary: mat.$violet-palette,
        theme-type: dark,
      ),
      // typography and density
    ));
  }
}

Thanks to a CSS media feature called prefers-color-scheme, we can achieve exactly the same behavior, only with much less compatibility issues, as this is supported back to iOS 13.

The reason for this is that if we define theme as 1-color, globally accessible properties like var(--mat-sys-primary) will not evaluate to a function holding both colors like light-dark(#005cbb, #abc7ff), but to color directly like#005cbb.

Please do (not) test in production!

After the time I invested in switching all the M2 code to M3, I was pretty disappointed to run into such a problem, but thanks to my great colleagues at Cloudflight, we were able to quickly find a good solution to this problem while still staying with Angular Material 19.

Although no users were harmed during this experiment, please, do not test in production. It's dark. But yeah, dark is not bad if the user requested it like that 😁😁😁

Thanks for reading.

This blog post aims to serve as both an introduction and a set of suggestions to help you manage such projects and customers regardless of whether you are a Requirements Engineer, Business Analyst or a Project Manager. On the other side, it aims to support customers with a bit of insight into how fixed priced projects are best handled, so that they can collaborate more effectively.

Before we get into it, a quick introduction is needed of the subject matter.

What does Fixed Priced mean?

A fixed price software project is a type of contractual agreement where the client and the service provider agree on a set price for the entire project before any work begins. This price remains constant regardless of the actual time or resources expended to complete the project. The scope, deliverables, and timelines are clearly defined and agreed upon in advance.

Pros and cons

There are several benefits and risks when getting into such a project:

Pros

• Customer defines scope in advance and knows exactly what they will receive

• Customer knows the timing of the delivery, and can plan work around that delivery

• If the estimation is accurate, the service provider knows how many resources they need to allocate

• Simple, low complexity projects with a small time frame are a great option as it helps keep the focus and achieve the end goal

Cons

• Delivery and value of the software is fully dependent on the quality of the predefined requirements, deliverable descriptions

• Little room for including additional functionalities that are discovered during development

• Fixed Price projects are usually handled with a Waterfall approach which has proven to be extremely risky, especially for the customer

• Cost estimation is fully based upon an initial understanding of the customer needs. During development this understanding may evolve, thus requiring more effort

• Delays have a compounding effect. If the project plan assumes the acceptance of a deliverable by a specific point of time which is not met, then the whole project timeline may be shifted

• The fixed scope nature makes it difficult to adapt to changes in the business environment

• Large scale projects have too many risks associated

After looking at all these pros and cons, several steps can be taken to manage them well.

Project Lifecycle

Below you can find several suggestions on what you can do to support the Project setting in the Project Lifecycle

Estimations

As with all software projects, an accurate estimation means a successful delivery, happy customer and a project that has stuck to the planned budget.

Depending on the scope size though, the estimation accuracy may be reduced because of which buffers have to be introduced to support the increasing risk size.

In order to manage that, the following measures can be taken:

• Splitting of the scope into multiple smaller scopes that can be more easily handled, estimated and delivered

• Separating the requirements of the customer into risk categories, and either descoping or closely managing high-risk requirements

• In case the scope is too big, and separation is not possible, additional budget can be negotiated for design and/or in-depth analysis of requirements be. This should provide with an increased insight into the scope to be able to do a more accurate estimation

• If none of the above is possible, it should be communicated to the customer that a safety buffer has to be applied to manage all the unforeseen risks.

A successful estimation is one where you, as Requirements Engineer, feel confident in having an extremely good understanding of what the scope entails and both you and the customer are satisfied with the budget.

Additionally to understanding the scope, consider effort for the following tasks:

• Project Setup

• Testing

• Meetings

• Documentation

• Clarifications, negotiations, defect and requirement analysis

In our experience this usually is between 30-50% of additional time, depending on the customer.

Design Stage

Once the customer has agreed to the Cost Estimation of the work scope, the Design can start.

During the design phase, alongside the usual design process of clarification, backlog planning, and user story preparation, it is crucial for an RE to document and clearly communicate what the customer will not receive. It's also important to record their agreement to this. This will make life during delivery exponentially easier for both parties.

As the requirements received from the customer are the cornerstones of the design, a core part of the Design Stage is making sure that there is a common agreement on what those requirements mean, usually in a documented form to further reduce potential misunderstandings, and also to crystallize the scope and design.

Depending on the quality of the initially drafted requirements, a process of rewriting, nomenclature alignment, and acceptance criteria polishing is advised.

The customer may or may not have a product owner that can support during planning. The luxury of Fixed Price, Fixed Scope projects is that the order of implementation is usually not imposed by the customer. This way there is room for prioritization not by value (as all requirements are equally valuable in terms of the contract), but by logical, technical dependencies to make implementation as streamlined as possible.

During the design stage it can also be a good idea to already discuss the process of the customer requesting additional scope. The requests will inadvertently happen once the customer starts using different draft versions of the application and realizes that the requirements have not exactly reflected what they wanted due to human error, interpretation issues, language barriers, etc.

In these situations it is important to

• Reinforce the customer and the willingness to provide the best software product

• Highlight the limitations of the contract

• Discuss next steps in order to be able to deliver the requested feature i.e. in a different contract or by descoping something else

• Refer to the process you have agreed upon in the Design Stage

Further details about handling scope changes will be explained below.

Delivery

Waterfall vs Agile

An assumption that is usually made when discussing the delivery of fixed price, fixed scope projects is that the customer has communicated everything, you have all the requirements available, thus customer involvement is not necessary until the full scope has been implemented.

As proven thousands of times, this approach of waterfall does not work in software projects due to the sheer complexity these products usually have.

If the customer wants to have exactly what they want, the way they want it, they will need to involve themselves actively during development. Usually, a Product Owner can support the development process with in-depth knowledge, which can significantly speed up decision-making. Alternatively, a representative of the end users should be available to the development team to assist in making informed decisions that best serve the customer.

Delivery Process

In the process of delivery, the usual Agile approach shall be taken with a few additional pre-steps and post-steps.

The usual Agile approach with Scrum consists of the four agile ceremonies: Sprint Planning, Daily Standup, Sprint Review, Sprint Retrospective. These should be enough to facilitate a healthy iterative delivery to fully handle and develop the backlog items.

Next to this, the following auxiliary meetings are advised:

• Before the sprint planning, review the scope of the requirements and clarify both what they customer will receive and what they will not

• A weekly meeting with the customer to discuss any additional open topics, questions or defects

• A weekly meeting with the decision maker on the defects, potential additional scope, project health

It can also have a great benefit to provide the customer as early as possible with mockups or small PoC solutions so that they can decide whether the approach you would like to implement would actually work for them. Help the customer make the decision by also highlighting what the consequences could be, both positive and negative, so they can make informed choices.

Defects Handling

After each iteration of delivery, it is expected that the customer tests the delivered requirements. During this testing process, defects can and will appear, that can be of the following categories:

• Defects that clearly don’t satisfy one or more requirements

• Defects that don’t satisfy the spirit of a requirement

• Defects that are additional scope

Whoever analyzes these defects, their first task should be correctly assigning the defect into one of these buckets first, then providing an appropriate answer.

• In the case of clear defects, they should be planned for a further Sprint

• In the case of grey area defects, a discussion shall be had on the value vs cost. If the cost/value seems too high, the defect needs to be descoped

• Defects that are additional scope, or new feature requests should be assigned to a later contract.

Although, intuitively we want to deliver the best software we can for our customer, and solve all their problems, there simply is not enough budget to be able to do that.

Contract Closure

Hopefully, by following a rigorous delivery approach, the delivery of the full scope within the budget was possible. Depending on the timeframe of the interaction, several outputs and outcomes were achieved, upon which you should be able to build:

Outputs:

• Documentation

• Software Code and functionality

• Testing supporting the functionality (either history of testing or automated test plans)

• Backlog of unresolved defects / tasks that have been defined out of scope

Outcomes:

• A very well defined and oiled delivery process

• Great interaction and involvement from the customer

• Understanding of additional effort needed next to the requirements

You can use these as inputs to the following steps:

• The backlog of defects/tasks can be an opportunity for a new contract, polishing the application, applying improvements

• Building upon the already existing processes they can rest assured that delivery will be quicker than with other providers

• Future estimations will be more accurate with the same customer

Closing Thoughts

Whenever working in such a restricted context, it is important to always be reminded of the rules of the game. Although the natural human behavior is to support the customer and make their lives easy, this may be counteractive to the goals of finishing on scope and in budget.

Being able to provide affordable alternatives and handle difficult situations in Fixed Price projects is the bread and butter of a requirements engineer. The customer may request something completely outside the scope, and it is essential to stay open minded and try to support them with ideas that can still fit into the scope. As a last resort there is always the option of saying a well framed “no” with the goal of reminding the limitations of the contract setting.

Agreements on processes should be made early to avoid surprises and negative emotions on either side.

Handling such projects can be tough but extremely rewarding if you can stick to the initial planning.

Cloudflight has a year of experience fine-tuning these models to make content retrieval for our customers more relevant. For example, we worked with the Austrian Press Agency (APA) on custom embedding models to improve knowledge discovery in the fast-paced news industry. In collaboration with the G39 group of European news agencies we are now making the developed system multi-lingual to allow users to retrieve relevant news in nine languages by querying in any language, making it easier to stay informed of reporting across different news agencies.

Multilingual semantic search isn't limited to news; it can also be useful in areas like international e-commerce or booking platforms. However, creating a truly multilingual system is challenging because models often prioritize results in the query's language. In this blog post, we aim to highlight this issue, explain it, and discuss how we mitigated it.

What is Language Bias?

Language bias occurs when multilingual semantic search systems prioritize results in the same language as the user query over results in other languages, even if the latter are more relevant. This bias is problematic, especially when the answer to a query is not available in the query language but exists in another language. In such cases, the system may return many irrelevant texts in the query language before showing the relevant ones in a different language. The term language bias was coined by Roy et al. (2020) in the paper LAReQA: Language-agnostic answer retrieval from a multilingual pool [2]. The authors define cross lingual alignment and describe two scenarios – weak and strong alignment:

Weak Alignment*: For any item in language L1, the nearest neighbour in language L2 is the most semantically relevant item.*

Strong Alignment*: For any item, all semantically relevant items are closer than all irrelevant items, regardless of their language. Crucially, relevant items in different languages are closer than irrelevant items in the same language.*

Graphic taken from LAReQA: Language-agnostic answer retrieval from a multilingual pool [2]

Weakly aligned multi-lingual embedding models work well, when we want to use the same embedding model for each language content separately (e.g. I only want to host one embedding endpoint but have many content databases in different languages) or when I want to search in my language (e.g. German) in a French content database without translating the query.

However, if you need to perform semantic search in a content database containing texts in different languages, only models with strong alignment will produce results that are free of language bias, thus making sure that the most relevant candidates are ranked highest and not the ones matching the query language.

Visualization of the embedding space

An easy way of detecting the presence of language bias in a multilingual model is to visualize the embeddings of a set of parallel sentences in two languages in 2D. That might look as follows, where the diagram shows embeddings of intfloat/multilingual-e5-large, a very popular multilingual open-source model with language bias.

The diagram shows the data distribution in the model, where German data is shown in red and English data in blue. We can see that language is greatly influencing the embedding distribution. A strong separation between the coloured clusters means that there is a language bias.

In contrast, the following diagram shows blue and red points well interleaved, which means English and German embeddings map to the same semantic space and are not easily distinguished by language.

Beware that this visualization technique is only an illustrative starting point and does not allow us to quantify the degree of alignment.

How to measure it?

To quantify and compare the language bias of different models, you can follow one of these approaches:

One option is to evaluate your model on the LAReQA dataset as described in the paper LAReQA: Language-agnostic answer retrieval from a multilingual pool by Roy et al [2]. Besides a method for quantifying the bias, the paper proposes a useful heat map visualization where you can see the degree of alignment between all language pairs. On the downside, the LAReQA dataset does not consist of a lot of languages.

As an alternative, you can follow the approach from Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation by Nils Reimers and Iryna Gurevych [1]. They propose to evaluate the model on a semantic textual similarity (STS) task using a multilingual STS dataset and see how much worse the model performs when you don’t test on every language individually but on a candidate set with all languages at once.

In addition to quantifiable measures, it’s also advisable to test for language bias manually. Select a couple of queries, translate them to different languages and issue a search for each translation. You should find approximately the same results for each translation of the same query. If you see results following the language of your query instead, you have a biased model without strong alignment.

Mitigating language bias

For the G39 project we built up large fine-tuning datasets from news articles from different countries to let the model learn different news contexts. Due to our experience with training mono-lingual embedding models we were confident that we could create a model that knows the current news context of multiple countries. However, another central question was whether this trained model would be usable for multilingual semantic search. We researched and brainstormed many ideas, but the solution was surprisingly simple in the end: the baseline training approach produced satisfactory results already. That is, we merged our nine large monolingual finetuning datasets together into one consisting of queries and news article chunks in nine different languages (but each query-article-pair still used only language). Training on this dataset using a standard contrastive loss with in-batch negatives eliminated most of the language bias that we measured and negatively noticed in the base model.

Why does this work? We are still in the process of finding out, but we can already share two contributing factors: Firstly, having a large amount of high-quality multilingual fine-tuning data at our disposal, and secondly, having a considerable topic overlap between the different languages in the dataset, because a good part of the news is international.

We nicknamed our approach “brute-force mitigation”, since obviously having a large high-quality amount of training data did the trick. If you don’t have large amounts of multilingual finetuning data at hand but maybe just data in one language, the approach from Reimers and Gurevych [1] may be worth taking a look at. The paper explains how to extend the capabilities of an existing embedding model to new languages with a training objective that targets equal embeddings in all languages (i.e. strong alignment).

References

[1] Nils Reimers and Iryna Gurevych, 2020, Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation https://arxiv.org/pdf/2004.09813

[2] Roy et al., 2020, LAReQA: Language-agnostic answer retrieval from a multilingual pool https://arxiv.org/abs/2004.05484

When it comes to test automation in general, it's easy to say that it's complex. If we were to stick to the classic separation of the Test Pyramid, we'd end up with:

- Unit tests, which can be implemented for both backend and frontend.

- Integration tests, which today can mean anything from API testing, microservices and third-party service integration, user authentication and authorization, to quasi-performance testing.

- End-to-end tests, which for the vast majority of test automation engineers and software developers would mean web application testing (to name a few major frameworks: Playwright, Cypress, Selenium).

There are also tools like Pact for contract testing, Storybook for advanced visual and/or snapshot testing combined with interactive documentation, and probably many more that I don't even know about.

However, these tools are well documented and have huge communities around them, and to be completely honest - I don't think any of these tests are hard (except maybe the contract tests).

When I started working on mobile test automation, I quickly realised that this was not the case. The more I got into mobile software development, the more confusing it became, the mere amount of dependencies and requirements to just get the application running was really bizarre, let alone automating tests on it. The documentation on the various technologies was sometimes not very helpful, and there were no articles on the full spectrum of possible mobile environments.

So let's fix that and talk about mobile test automation, shall we?

Development Frameworks Overview

First, before we go any further, we need to consider what technologies will be used to build the applications that will be tested with the mobile testing frameworks. The popular frameworks for mobile development are:

According to the 2024 Stack Overflow survey, Flutter is the most popular framework for cross-platform development. It's followed by React Native, Electron (not listed in our table because it only supports desktop app development), .NET MAUI (and it's older brother Xamarin, which has been replaced by MAUI), Ionic, and Cordova closing the list. Interestingly, there is no mention of NativeScript at all.

Mobile Test Automation Frameworks Overview

Below is a general overview of the test automation frameworks. There are some other frameworks that I have intentionally left out because they do not seem to be very popular (e.g. EarlGray, which is Google's framework for iOS automation), and they are most likely wrappers of the technologies listed in the table anyway (e.g. Selendroid or any of the cloud mobile test automation providers).

Appium-based Frameworks Overview

From the above list, Appium can be considered as the most comprehensive and therefore the most complicated framework of them all. In short, it consists of four basic elements: Appium Core, Drivers, Clients and Plugins.

We won't go into the details of what each element stands for, but to give you a better understanding of what technologies are supported by the Appium framework, here is a list of so-called Clients:

Scenario-Based Guide to Selecting Mobile Testing Frameworks

When you would have to make one of the key decisions in the context of testing strategy, that is, the choice of a framework for automating mobile end-to-end testing the choice is quite complicated, as it depends on many factors.

Let's do a little thought experiment and try to describe some possible scenarios to help you decide.

TL;DR:

• When developers are using native development tools and want to maintain grey/white box end-to-end testing and/or you’re building an MVP: Espresso (for Android) / XCUITest (for iOS)

• If you're developing an MVP and you only need UI testing (without physical iOS devices and with limited scripting possibilites): Maestro

• If you're developing a cross-platform React Native application and want to improve test scripting experience with Large Language Models: Detox

• For long-term projects that require stable solutions: Appium-based platforms

Scenario 1: Android or iOS Only

The first scenario is an example for likely early stages of development (e.g. seed stage of a startup) when the focus is on only one mobile platform. This means that the natural choice should be either Espresso (for Android) or XCUITest (for iOS).

However, there may be other things to consider, such as:

1. Resource-Constrained Environments:

For example, if the application is being developed on a tight schedule and/or budget, and the developers simply do not have the time to maintain the end-to-end test suites, choosing complex Espresso or XCUITest frameworks can end up being a disaster.

In this type of environment, I would suggest using Maestro as it seems to be the most hassle-free and does not require too much time and effort to maintain test suites and configuration.

2. Future Proofing for Long-Term Projects:

If your project's time and budget constraints are not the things that keep you up at night, then you should probably think a little more about the potential development path.

I want to emphasise that we should avoid over-engineering, but on the other hand, it's also important to choose a framework that we can use in a more mature test environment. By a more mature test environment I mean things like automated test data generation and cleanup (which can be achieved with API-based model) or implementation into the existing CI/CD pipelines. And of course we want these things to be relatively easy to implement.

In summary, if you are sure that your project is entirely Android or iOS based, developers are the ones who will write and maintain the test suites, and you do not plan to extend it to the opposite technology, I would suggest using Espresso or XCUITest, as they will give you the most solutions adapted to your existing codebase. Otherwise, I'd lean toward the Appium-based solutions because they're much more extensible and can be used in black-box environments.

Scenario 2: Android and iOS

Most of the time, companies want to target the broader audience, and since Android has ~72%* of the market share, which means that iOS users have ~28%* of the market share, it means that most companies want to target both platforms.

This scenario, at least from my perspective, is much more complex in terms of things that need to be considered before an informed decision can be made.

1. Separate development teams for Android and iOS

   

If your application is developed by separate teams using native development tools (i.e. Android Studio and/or Xcode), you can probably integrate Espresso and XCUITest to perform end-to-end testing. However, I would like to point out that this may be a subpar solution, as other frameworks allow you to test both platforms using the same test scripts.

So, for the situation mentioned in this sub-scenario, I'd suggest using Espresso and XCUITest for smaller scale tests (component testing if you will) that would be maintained by the developers themselves and integrating broader scoped frameworks for end-to-end testing (i.e. Appium or Maestro).

2. Cross-platform development

   

As you can see in the table mentioned in the Development Frameworks Overview section, there are many possible frameworks to choose from to develop cross-platform applications. Most frameworks support all platforms (mobile, web, and desktop), but when it comes to mobile development, we focus mostly on Android and iOS.

Unfortunately, in order to figure out which framework we should choose to write and maintain end-to-end mobile tests, we need to proceed with the sub-categorisation of possible needs and settings.

• If you really don't have the time to maintain test suites, and therefore need something simple, almost effortless to write and maintain, I would suggest going with the Maestro.

• If you are an SDET or an experienced test automation engineer, you may want to lean more towards Appium-based tools, as Appium gives you all the benefits of mobile test automation as well as the freedom to configure the entire test automation framework yourself. I'm not saying that Maestro can't do this, but please take a look at the Maestro Afterthoughts for more insight.

3. Cross-platform development with React Native

   

The tool I haven't mentioned yet is Detox. This is because Detox is specifically designed for grey-box end-to-end testing for React Native. I'd recommend giving it a try if you're developing a React Native application, especially given the fact that it's the first known framework to incorporate the Large Language Models to write tests in natural language.

However, if you and your team are developing applications for external customers (so-called enterprise-grade applications) that require stable tools, I'd strongly recommend using Appium-based tools because they are much better documented and have much larger communities built around them.

Scenario 3: Android, iOS Web and More

The last scenario we'll talk about is native mobile development combined with any other version of the app. This is paradoxically the easiest of all.

Appium.

There is simply no other solution that would allow you to test on all of the available platforms.

Of course, if you'd like, you can mix and match Maestro for mobile testing and Playwright for web testing. Or Detox for mobile, Appium for smart TV, and Selenium for web, if that's what you want. It just doesn't seem like a good idea, especially since some of the Appium-based frameworks allow you to create a single test script that would run sequentially on multiple platforms.

For example, WebdriverIO allows you to do something on one mobile device, then switch to a separate web application to change/verify values and perform some other actions on another mobile device.

Maestro Afterthoughts

I felt that Maestro might seem like a great solution in the end, which it can be in some cases, but I also want you to think about possible downsides if you decide to go with it.

Maestro gives you the ability to configure the test automation framework (e.g. run JavaScript code), but since it's written with a specific architecture in mind, it's more of an add-on to the standard YAML flow files than anything else.

Another thing to keep in mind when talking about the Maestro is last year's Cypress intellectual property update case. In a nutshell, the Cypress team removed the ability to use third-party services for dashboards that were previously free, essentially forcing companies to use their own cloud platform.

Not only is that platform not free, but it also prevents Cypress from being used where contracts with customers specifically block the use of cloud platforms due to GDPR or similar laws. I mention this because Maestro has its own cloud platform for running tests called Robin, and it's possible that the similar scenario to the Cypress update could happen to Maestro as well.

Development for the Apple ecosystem

The last thing I wanted to mention here is that if you want to write tests for the Apple ecosystem, namely iOS, iPadOS, macOS, or tvOS, you actually need to own a machine based on macOS (either a MacBook, Mac Mini/Studio/Pro, or iMac) because you need to have Xcode installed.

The reason is simple - it contains the Simulator application, which allows you to run your application in the virtual environment and is required to run tests on real devices. You'll also need an Apple Account, and if you want to publish your application on the AppStore, you'll need to sign up for the paid Apple Developer Program.

Quantization

Quantization is the process of decreasing the computational precision of a model to a lower datatype, e.g. FP32 (floating-point) to INT8. It is strongly dependent on the hardware, which datatype you should prefer while quantizing. In this article, we will mainly focus on INT8 for the following reasons:

• Coral Ai Edge TPU only supports INT8 operations

• NXP iMX8 Plus is more efficient in INT8 precision and consumes less power

• Newer Intel CPUs have dedicated INT8 co-processors and instruction sets i.e. Intel® Advanced Matrix Extensions, which can do 2048 INT8 operations per cycle

What do we expect from a quantized model on dedicated hardware?

• Efficient execution in both performance and power consumption

• Decreased model size:

  • Coral Ai Edge TPU has only 8MB of cache

  • Newer Large Language Models are even quantized to INT4 to be able to fit into consumer GPU hardware

• Decreased model accuracy, due to lower computation precision

How to quantize a model?

• Post-training quantization, where an already trained model is quantized. Each edge device normally has its quantizer to deploy models.

• Quantization aware training, where during training not only the model accuracy but also the quantization factors are optimized, which increases the complexity of training and deployment. If necessary, activation functions can also be modified according to hardware requirements.

How does quantization work?

• Scaled Integers, where real numbers are represented as integers multiplied by a factor and shifted with a bias

• Lookup table (LUT), complex functions like sigmoid and softmax can be precomputed and their outputs can be stored

• When these methods are applied a representative dataset is required

• There exist other techniques, but these are the most common to be applied

Deploying YOLO

Now that we have a better overview of quantization let's try to quantize the YOLO model. To continue, we need to have a high-level understanding of the YOLO model:

• Backbone, which is a convolutional neural network and outputs a feature pyramid

• Head #1, where the last 3 layers of the pyramid are upscaled and convolved

• Head #2, where the Detection layer is executed

• Head #3, where the scores and bounding box coordinates are computed

Fixing the Head #3 of YOLO

When applying INT8 quantization on the YOLO7 model, the output values were corrupted. While analyzing the code, it was observed that pixel coordinates and the class score values are not in the same range, therefore quantization collapses. After applying normalization on the coordinate values by image size, a more reasonable output was visible, but still, some values were wrong. Models with large input image sizes were still suffering from quantization collapse. After analyzing the model quantization parameters (factor, bias) it was found that a large numerical instability is present. This was further improved by using normalized scaling factors instead of normalizing output values by the image size. Precomputing the scaling factors is possible because we quantize static models, meaning the input size must be always the same and cannot change during inference.

As the next step, YOLO8 was tested, where the same numerical instability was visible. Changing here to use also normalized scaling factors instead of normalizing by image size solved a bug and it further improved the model precision by 4% compared to the previously reported values.

We can observe this on the first image, where normalizing with image size results in a factor difference of 10^5.

Where on the second image we see that using normalized scaling factors for bounding box calculation our quantization factor difference is only 10^2.

Should quantization be applied to Head #3 at all?

Tests have shown that if the last mathematical operations are excluded from the quantization the precision of the model increases, while the processing time is barely increasing. On a device with only INT8 operations, this would mean that the final steps need to be executed on the CPU, if FP32 instructions are available. This would also mean that normalization is not necessary and the previous discussion would be irrelevant. Detaching the head is at the moment rather cumbersome to implement for TFLite quantization (TPU, NXP) and only can be achieved with dirty hacks.

For Intel CPUs, this is a different scenario as operations can fall back on the CPU to be executed in FP32 precision. OpenVINO quantizer supports such changes by explicitly defining which operations should be excluded from the quantization process. Due to this freedom, normalizing is not necessary as the quantizer implicitly excludes operations if quantization collapses. After careful testing to exclude the whole Head #3, the precision of the model has been improved by 1.4% while the average inference speed only increased by 0.3% on an Intel 9th Gen CPU.

NOTE: OpenVINO applies per-channel quantization, while TFLite can be switched between per-tensor or per-channel. Per-tensor has one factor and bias, while per-channel has for each channel a factor and bias.

Activation Functions on Edge Devices

Since some of the selected devices have restricted instruction sets, different activation functions are needed for deployed models.

• LeakyReLU

  • YOLOv7-Tiny is trained with LeakyReLU

  • Coral Ai Edge TPU does not support LeakyReLU

  • NXP output is corrupt using LeakyReLU

  • Intel works with LeakyReLU

• SiLU

  • YOLOv8n is trained with SiLU

  • Coral Ai Edge TPU crash using SiLU

  • Intel works with SiLU

• ReLU6

  • ReLU6 achieves lower mAP after training for both YOLO7 and YOLO8

  • ReLU6 has less accuracy drop after quantization

  • ReLU6 works both on Coral Ede TPU and NXP

Final Thoughts

Quantization is a hardware-dependent task. To achieve the best results, one should understand both the AI model and the hardware to be deployed. The changes about TFLite and OpenVINO mentioned in this article regarding YOLO8 have been merged into the main repository.

State of the AI

In recent years, there has been a huge advance in AI with the introduction of Large Language Models. These methods have also been adopted by the Computer Vision community, where images are converted to tokens and passed to a Language Model for classification, detection, or other tasks, these are called Vision Language Models. Although these methods show better performance in regards to accuracy or precision, they require far more processing power compared to their older relatives, let's call them the Convolutional Neural Network ones. The need for processing power becomes an even larger issue when deploying a model on the edge.

• Large Language Model (LLM), e.g ChatGPT, Llama2

• Vision Language Model (VLM), e.g GPT-4V, Swin-L or ViTL

• Convolutional Neural Network (CNN), e.g YOLO or EfficientNet

Object Detection

Object detection is a task in computer vision, where we take an image as an input and localize and classify objects within the input image. The de facto metric to assess the quality of an object detector is the mean Average Precision (mAP). This is calculated by the sum of the intersection between the predicted bounding box and the annotated bounding box.

YOLO (Ultralytics on GitHub)

YOLO (You Only Look Once) is the state-of-the-art convolutional neural network based method for object detection, but the newer YOLO versions can perform other tasks like segmentation, pose estimation, or classification.

As we see from the table above VLM methods outperform CNN methods, but require a higher number of trainable parameters. The number of parameters is an indication of both computational power and memory usage, but multiple factors influence inference speed. YOLO comes also in different sizes, indicated by the last character in the name. The smaller a model is, the faster the inference is, and the smaller the memory consumption is, but also the accuracy decreases.

Edge Devices

Edge devices are limited by processing capabilities and power consumption. They give AI capabilities for IoT devices, sensors, cameras, drones, and smartphones. There are low-end solutions that can cost 20$, but one can also choose from high-end System-on-Chip (SoC) solutions above 1000$. The rule of thumb is if it costs more then it has higher processing power and more power consumption. When choosing the hardware, there will be a tradeoff between price and performance. The following tables show a few examples of devices as of February 2024.

Deploying a quantized YOLO

In this post, we assume that we already have a quantized model. In short, quantization is the step where the model precision is decreased, for example instead of using FP32, we quantize to INT8. While on many hardware this comes as an improvement in inference speed, the accuracy of the detection will decrease. We discuss quantization in more detail in the next chapter, Quantized YOLO for Edge Solutions.

Model deployment on edge hardware is different for each device. Let's discuss a few cases:

• Coral AI works only with INT8 precision. This means the model weights are in INT8 precision and inference is performed as integers.

• NXP is most efficient also using only INT8 precision.

• Intel can execute inference in different ways. It also depends on, which generation CPU is being used. In general, it consists of a CPU where you can execute either in FP32 or combine FP32 with INT8, an iGPU where you can execute in FP16, and an NPU where you can execute only in INT8.

Now let's test the inference speed on the following hardware:

• *Intel i7-9750H

• **Raspberry Pi4 + Coral AI Edge TPU

The Setup

This is a home-made setup, which can detect pigeons in almost real-time. This is a Raspberry Pi4 connected with a battery pack and 2 Coral Ai Edge TPUs. One for bird detection and the other for bird classification. After deploying this setup, together with two plastic crows, no more pigeons landed on my balcony in the last 1.5 years. Check out my repo to make sure no pigeons make your balcony dirty. GitHub Repository

For more technical details on the quantization, continue reading the next chapter, Quantized YOLO for Edge Solutions

Today we will focus on the interaction modes and highlight the possibilities in the context of a test automation enabling team.

Unlocking Interaction Modes: Empowering Enabling Teams

In the dynamic landscape of software development, test automation has emerged as a cornerstone for achieving speed, efficiency, and quality in product delivery. Within this ecosystem, the role of a test automation enabling team is pivotal. These teams specialize in providing the necessary expertise, tools, and support to empower other teams in implementing and maintaining robust automated testing practices. However, the success hinges not only on their technical proficiency but also on the effectiveness of their interactions with other teams.

In the next sections, we will explore three key interaction modes - Collaboration, X-as-a-Service, and Facilitation - and identify which modes are best suited to maximize their impact.

Collaboration: Fostering Shared Responsibility

Collaboration lies at the heart of effective teamwork and is particularly vital for test automation. By adopting a collaborative approach, the automation team can forge strong partnerships with development, testing, and operations teams, working together towards common goals.

• Shared Understanding: Test automation enabling teams collaborate closely with other teams to gain insights into their testing needs, challenges, and priorities. By fostering open communication channels, they ensure that automated testing efforts are aligned with overall business objectives.

• Cross-functional Expertise: Leveraging the diverse skill sets within the organization, collaboration enables test automation enabling teams to tap into domain-specific knowledge and technical expertise. This cross-functional collaboration enriches the quality of automated tests and enhances the effectiveness of testing strategies.

• Continuous Improvement: Through collaborative retrospectives and feedback loops, teams can reflect on their testing practices, identify areas for improvement, and iterate on their automation strategies. By fostering a culture of continuous learning and adaptation, collaboration drives innovation and excellence in test automation.

X-as-a-Service: Providing Scalable Solutions

Providing testing-related services as a centralized offering.

• Standardize Practices: Through X-as-a-Service, test automation enabling teams can establish standardized testing frameworks, tools, and processes that can be leveraged across the organization. This standardization promotes consistency, reduces duplication of effort, and accelerates the adoption of automated testing practices.

• Scale Resources: Test automation enabling teams can scale their resources and expertise to meet the fluctuating demands of different teams and projects. By offering testing services as a scalable resource, they ensure that teams have access to the necessary support and guidance to accelerate their testing efforts.

• Enable Self-Service: X-as-a-Service models empower teams to become more self-sufficient in their testing endeavors. By providing self-service platforms, tools, and documentation, test automation enabling teams enable other teams to autonomously create, execute, and maintain automated tests, thereby reducing dependencies and promoting agility.

Facilitation: Guiding and Empowering Teams

Facilitation plays a crucial role in guiding teams through complex challenges, fostering collaboration, and promoting innovation.

• Clarify Objectives: Facilitation helps teams align on testing objectives, priorities, and strategies. By facilitating workshops, planning sessions, and brainstorming exercises, test automation enabling teams can ensure that testing efforts are focused and aligned with business goals.

• Resolve Conflicts: Inevitably, conflicts may arise during the testing process. Facilitation techniques such as mediation and consensus-building can help teams navigate conflicts effectively, fostering a positive and constructive working environment.

• Empower Teams: Facilitation empowers teams to take ownership of their testing processes and outcomes. By facilitating knowledge-sharing sessions, communities of practice, and peer learning initiatives, test automation enabling teams cultivate a culture of empowerment and collaboration.

Team API

Team API is a concept, referring to the interface or interaction points between teams within a project or an organization. Just like how software systems have APIs (Application Programming Interfaces) for communication between different software components, teams also need clear interfaces and communication channels to collaborate effectively. The Team API defines how teams interact, communicate, share information, and collaborate on work. It encompasses aspects such as responsibilities, expectations, communication channels, decision-making processes, and dependencies between teams. Establishing clear and well-defined Team APIs helps to streamline collaboration, reduce misunderstandings, and improve the overall efficiency of the organization. However, it is just as important to regularly challenge and maintain these definitions.

Here you can find the official Team API template.

Our Journey - Success Through Effective Interaction Modes

In our journey as a test automation enabling team, embracing Team Topologies alongside Lean Coffee sessions, regular Meetups, and the introduction of a Team API has been transformative. Team Topologies provided us with a structured framework for organizing our teams and optimizing interactions, leading to improved collaboration and productivity.

By leveraging collaboration, X-as-a-Service, and facilitation, we can orchestrate success, driving continuous improvement, innovation, and excellence in automated testing practices. Striving to deliver high-quality software at speed, the strategic adoption of interaction modes in team topologies emerges as a critical enabler for achieving this ambitious goal.

Understanding Team Topologies

Team topologies, introduced by Matthew Skelton and Manuel Pais, provide a framework for designing effective team structures within an organization. It emphasizes the need for teams to align with the organization's architecture and business objectives. The four fundamental team types are:

• Stream-aligned Team: These teams focus on delivering value directly to the customer. They are cross-functional and have all the necessary skills to deliver end-to-end solutions. Stream-aligned teams own a specific part of the business or product, enabling faster decision-making and reducing dependencies on other teams.

• Enabling Team: Enabling teams provide support, tools, and platforms to streamline the work of stream-aligned teams and acting as a catalyst. They focus on creating self-service platforms, automation, and providing expertise to enable other teams to deliver efficiently.

• Complicated Subsystem Team: Some parts of the system are inherently complex and require specialized knowledge. Complicated subsystem teams are responsible for maintaining and evolving these parts of the system. They collaborate closely with stream-aligned teams, providing expertise and guidance when needed.

• Platform Team: Platform teams build and maintain shared platforms and services that streamline the work of other teams. They focus on creating reusable components, APIs, and tools that increase productivity and consistency across the organization. Platform teams abstract away common functionality, allowing stream-aligned teams to focus on delivering value.

These team topologies are designed to foster effective communication (there will be a follow-up article, where the communication and interaction models will be discussed in more detail), collaboration, and autonomy within organizations. By aligning teams with the value streams of the business and providing the necessary support and expertise, organizations can improve their agility, speed, and ability to innovate.

Enabling Teams and Test Automation

Enabling teams play a crucial role in fostering collaboration and enabling other teams to succeed. When it comes to test automation, enabling teams can significantly impact the efficiency and effectiveness of testing efforts across the organization. Here's how:

• Providing Test Automation Frameworks: Enabling teams can develop and maintain robust test automation frameworks tailored to the organization's needs.
  These frameworks can include libraries, tools, and best practices for writing and executing automated tests.

• Offering Training and Support: Enabling teams can offer training sessions and workshops to educate other teams on test automation best practices and tools. They can also provide ongoing support and guidance to help teams overcome any challenges they encounter during test automation implementation.

• Integrating Testing Tools: Enabling teams can integrate testing tools into the development workflow, making it seamless for teams to incorporate automated testing into their processes. This integration can include CI/CD pipelines, version control systems, and issue tracking tools.

• Promoting Collaboration: Enabling teams can facilitate collaboration between development and operations teams to ensure that test automation efforts are aligned with overall business goals. They can encourage cross-functional collaboration and knowledge sharing to improve the quality of automated tests.

Case Study

Implementing Test Automation with an Enabling Team. Let's consider a hypothetical scenario where a software development company decides to implement test automation with the help of an enabling team.

Challenges

• Manual Testing Overload: The team is overwhelmed with repetitive manual testing tasks and regression testing, leading to slower release cycles and increased risk of human errors.

• Inconsistent Testing Practices: There are inconsistencies in test execution, test coverage and reporting across different projects.

• Lack of Automation Expertise: The development teams lack the expertise and resources to implement and maintain test automation frameworks effectively.

A (possible) Solution

To address these challenges, the software development company decides to form an enabling team dedicated to implementing test automation across the organization. The enabling team consists of experienced automation and quality engineers, and DevOps engineers who collaborate to build robust automation frameworks and provide support to development teams.

Implementation Steps

• Assessment and Planning

  • The enabling team conducts a thorough assessment of existing testing processes, tools, and infrastructure.

  • They identify areas where automation can bring the most significant benefits and prioritize them based on impact and feasibility.

• Framework Development

  • The enabling team designs and develops a flexible and scalable test automation framework tailored to the company's specific needs and technologies.

  • They choose appropriate testing tools and technologies based on the project requirements and industry best practices.

• Training and Support

  • The enabling team conducts training sessions and workshops for development teams to educate them about test automation best practices, tools, and frameworks.

  • They provide ongoing support and guidance to help teams adopt automation seamlessly and address any challenges they encounter during the transition.

• Integration with CI/CD Pipelines

  • The enabling team integrates test automation scripts into the company's continuous integration and continuous delivery (CI/CD) pipelines to automate the execution of tests as part of the build and deployment process.

  • They implement reporting and alerting mechanisms to provide real-time feedback on test results and identify issues early in the development lifecycle.

• Monitoring and Maintenance

  • The enabling team establishes monitoring mechanisms to track the performance and effectiveness of automated tests.

  • They continuously monitor and maintain the automation framework, updating it as needed to accommodate changes in the software and technology landscape.

Results

• Increased Test Coverage: Test automation significantly increases test coverage, allowing for more thorough testing of critical functionalities and edge cases.

• Faster Release Cycles: Automated tests reduce the time spent on manual testing, enabling faster and more frequent releases without compromising quality.

• Improved Quality: Automation leads to fewer defects in production, resulting in improved customer satisfaction and reduced support and maintenance overhead.

• Empowered Teams: Development teams feel empowered to focus on delivering value-added tasks, knowing that repetitive and time-consuming testing activities are automated.

• Continuous Improvement: The enabling team fosters a culture of continuous improvement, regularly evaluating and enhancing the test automation practices to keep pace with evolving technology and business needs.

Conclusion

In conclusion, leveraging enabling teams can greatly enhance test automation efforts within an organization. By providing frameworks, training, support, and promoting collaboration, enabling teams can empower other teams to implement and maintain effective automated testing practices. As organizations continue to prioritize quality and efficiency in software development, the role of enabling teams in driving successful test automation initiatives will become increasingly important.

The series will continue with the next part focussing on communication and interaction patterns soon - stay tuned!

Key Takeaways

Data access restriction is important.

Do not use data masking as a security measure.

Create reports based on shared semantic models.

Starting Point

Let's assume we look at our Microsoft Fabric platform. We see fancy data pipelines that grep data from whatever sources you can think of, transform the hell out of the data leaving us with perfect, golden data inside a Lakehouse. Of course in form of nice delta tables.

Now how can we give data access to people within our organization and ensure that they

• Only see tables that are relevant for them? (Object Level Security)

• Only see columns that are relevant for them? (Column Level Security)

• Only see data (rows) that is relevant for them? (Row Level Security)

Basically how can we enable them to query the data they are allowed to see, understand the data model and build reports on their own? Just in case you need it, the fancy buzzword is Fostering self-service reporting while safeguarding data.

Strategy

To realize this goal we adopted the following strategy:

1. Read-Only SQL Connection: Users are granted read-only access to the SQL endpoint of the lakehouse. By default, they are shielded from access to underlying tables, minimizing the risk of inadvertent alterations or unauthorized access.

2. Row-Level Security (RLS): RLS mechanisms are deployed to dynamically filter data at the row level based on user identities or membership in Microsoft 365 groups. This granular access control ensures that users only interact with data relevant to their designated roles or responsibilities.

3. Column-Level Security (CLS): Complementing RLS, CLS configurations restrict access to specific columns within tables, safeguarding sensitive information from unauthorized disclosure.

4. Data Masking: As an additional layer of defense, data masking techniques are employed to obfuscate sensitive information displayed to users. Primarily cosmetic, this approach must not be compared to CLS as the risk of data leakage via brute-force queries is a given fact. See example lateron.

5. Semantic Data Models for Self-Service Reporting: Empowering users with semantic data models serves as a structured abstraction layer, enabling them to navigate and interpret complex datasets intuitively. This approach fosters self-sufficiency in report creation while ensuring adherence to organizational data standards.

Share SQL Endpoint

Sharing the SQL endpoint of a lakehouse with recipients or groups is straight forward.

To give recipient a basic Connect permission like in an SQL server be sure to NOT select any of the available sharing options. By doing so per default no data is available to be read and GRANT permissions need to be explicity defined. Some kind of a trust nobody strategy.

For more details see MS Data Warehouse Sharing.

Row-Level Security

Row-level security ensures that user can only access rows they are allowed to see.

These restrictions are applied directly inside the data tier (database) every time data is accessed. This makes it more reliable, robust and less error prone than restrictions that are applied inside an application tier.

To enable row-level security two things are needed

• a schema containing an inline table-valued function

• a security policy using the created function as a predicate

Let' have a look at a simple example.

First creating a new schema including a function that returns 1 when the value of the email is the same as the user executing the query. The value of the user is read via USER_NAME(). To make it more robust all values are converted to lowercase.

CREATE SCHEMA sec;
GO

CREATE FUNCTION sec.customers_validate_rlssecurity(@email as VARCHAR(500))
    RETURNS TABLE
WITH SCHEMABINDING
AS
    RETURN SELECT 1 AS validate_rlssecurity_result
    WHERE LOWER(@email) = LOWER(USER_NAME())
GO

Now all that is left to do is create a security policy, make the connection to the corresponding table (dbo.customers) and set its STATE to ON.

CREATE SECURITY POLICY CustomersFilter
ADD FILTER PREDICATE sec.customers_validate_rlssecurity(Email)
ON dbo.customers
WITH (STATE=ON);
GO

During development it can be useful to disable the policy temporarily. Just set the STATE to OFF.

ALTER SECURITY POLICY CustomersFilter
WITH (STATE = OFF);

Looking into performance of RLS is most likely an additonal blog post on its own. Roughly the impact will be comparable to using a view. Two simple, basic recommendations to keep in mind

• Keep the predicate function simple. The more joins the worse the performance.

• Ensure there are indices on referenced tables.

Column-Level Security

Nothing fancy here. Grant access to users, groups as needed.

GRANT SELECT ON customers(FirstName, LastName, Email) TO [restricted.robert@cloudflight.dev];

It is important to know, that CLS in place will prevent restricted users from executing SELECT * statements like the one below as they will fail with an error message. Wanted columns need to be specified explicitly in the SELECT statement.

SELECT * from customers;

Msg 230, Level 14, State 1, Line 12
The SELECT permission was denied on the column 'Age' of the object 'customers', database '****', schema 'dbo'.

In a way this may be seen as kind of leaking information as it tells the user that there are further columns and how they are named BUT at least they are not accessible.

It does however result in the direct SQL endpoint not being usable when creating a Power BI report. It will fail connecting to the endpoint with the same exceptions as above.

This issue can be solved by creating a semantic model of the lakehouse and use it as the base for creating a report. We will come to that later. Moreover it is the recommended approach when speaking about performance.

For more details see MS Column-Level-Security.

Data Masking

The main use case of data masking is to hide sensitive data in the result of a query.

It can be applied easily on columns by using a command like

ALTER TABLE dbo.customers;
ALTER COLUMN City ADD MASKED WITH (FUNCTION = 'default()');
GO

Resulting, as expected, in queried data being masked.

BUT there is a - from our point of view - rather alarming fact. One should never every use data masking as a replacement for column-level security. Data masking is applied once the data has been queried. This makes it possible to perform (automated) brute force queries for gathering insights into the masked data.

This is how - although data masking is in place - you can find out users from the marvelous, imaginary city Burnsmouth.

With having proper column-level security in place the user would not be able to get such insights as access to the city column would be denied completly.

Fore more details see MS Dynamic Data Masking.

Building reports with semantic models

Now that everything is secure, how can we best share data - including an easy to grasp data model - with someone eager to create reports?

The best way to achieve this is by sharing semantic models.

Let's look at some of the reasons why.

✔️ Firstly we have seen above, that with column-level security in place Power BI will fail accessing data from just an SQL endpoint as it is unable to query the structure of the tables. Most likely Power BI is doing a SELECT * FROM table to get all available columns which fails with proper column-level security in place.

✔️ Secondly a semantic model is a logical description of an extract of your lakehouse, most likely from a specific domain (e.g. Sales, HR). Most of the times it will be a star schema and can help report creating users easily understand the data model.

✔️ Thirdly the default connection type when using semantic models is the so called Direct Lake mode. It combines the existing connection types - Direct Query and Import - by combining their advantages. This means it is fast and there is no need for periodically updating your reports dataset to reflect the latest data changes. Going into detail would be a blog post on its own. For now you can find out more details here.

Every lakehouse creates a default semantic model but the recommendation is to create specific ones for individual use cases, narrowed down to the tables, relationships and columns the users need and are allowed to access (combined with column-level security).

Let's illustrate this process.

• Create a semantic model for a lakehouse

• Include needed tables and model their relationships

• Hide all columns that are not visible to the user because of column-level security (everything but the employee name from the dimension table)

• Share the semantic model with a group of users

• Let them import the semantic model in Power BI to build reports on top of it

After connecting we see the expected result. The employee table only shows the employee column. Job done.

Remember, with column-level security in place connecting to the SQL endpoint would fail during import.

That is it for now.

Stay tuned for more posts in the area of data engineering with Microsoft Fabric. Especially the important topic of how to automatically test granted permission and security in general might be an interesting next read.

Introduction

Software development is a time-intensive task and requires skilled software engineers to get the job done. Time and budget are directly proportional to one another and therefore as little development resources as possible should be wasted. Especially in small to mid-sized projects most development time should be spent on feature implementation and time spent on other tasks should be minimized.

As part of the rapid development project of the Cloudflight Technical Lab, we researched software solutions with a focus on quickly achieving results and developing a functional MVP in a very short time. Every team was tasked to develop a generic resource management system as described in the introduction of this article series. Our team specifically built an interactive React-Admin frontend and a NodeJS backend using Fastify with a PostgreSQL database. Our development insights are discussed in this article and the technologies are evaluated in terms of their suitability for rapid development.

Tech stack

Besides the general resource management functionality, we implemented the sending of automated confirmation e-mails in the backend using Mailhog as a test mail server.

React-Admin

React is a powerful web development framework with industry-wide adoption due to its clever component architecture and its capabilities to build interactive web pages. React-Admin builds on these attributes and adds useful extensions to simplify development and ensure maintainability. It is the promising bridge between custom high-code solutions and low/no-code solutions by reducing development time through abstracting existing React libraries.

The main benefit of React-Admin is its inclusion of numerous out-of-the-box Material UI components, which can be used to quickly build dashboard-styled web apps similar to YouTube Studio and Spotify. It also offers integrations to quickly implement authentication, permissions, internationalization and lots of other useful functionalities. React-Admin is best used for CRUD-based applications but can be limiting when developing complex web apps.

import {Admin, Resource} from "react-admin";
import {SpotList} from "../components/spots";
import {dataProvider} from "../providers/dataProvider";

export const App = () => (
    <Admin dataProvider={dataProvider}>
        <Resource name="spots" list={SpotList}/>
    </Admin>
);

The core of React-Admin - the data provider specification, the declared resources (only one in this case) and the Admin component

The SpotList is automatically fetched via DataProvider and all data is displayed in a table.

Pros

• React-Admin components are a good base for dashboard-style apps

• Components are styled and responsive by default

• Routes for resource endpoints are configured automatically

• Data provider abstraction allows for automatic API communication

• Built-in role-based access control

• Integrated internationalization (40+ locales supported)

• Scalability assured by React's architecture

Cons

• Custom functionality (everything besides CRUD) can be difficult to implement

• Backend API implementation has to follow data provider specifications

• Some features and UI components are only available via an Enterprise Edition subscription (e.g. site-wide search, breadcrumb paths, AI autocomplete and more)

• Little TypeScript documentation is available, mostly JavaScript

Fastify

Fastify is a modern NodeJS web framework that focuses on developer experience, low overhead and responsiveness by efficiently managing server resources. Its powerful plugin architecture allows developers to quickly implement features while requiring minimal configuration effort.

Plugins can be self-developed, but there are also core and community plugins, which can be added as dependencies. As an example, a “database connection plugin” can be registered in the application context and by using decorators it can then be accessed from anywhere in the application.

As a web framework, Fastify provides an easy solution to declare endpoints and routes for HTTP communication. Hooks can be used as event listeners to execute custom code, whenever a specific action is executed, e.g. a reply header is added before a request is returned.

const fp = require('fastify-plugin');

module.exports = fp(function (fastify, opts, done) {
    fastify.addHook("onRequest", async function (request, reply) {
        reply.headers(
            {"Access-Control-Allow-Origin": "*",
             "Access-Control-Allow-Headers": "*",
             "Access-Control-Expose-Headers": "*"}
        );
    });
    done()
})

A simple plugin that adds CORS headers to every reply

Moreover, Fastify requires very low overhead compared to similar frameworks. Only the core configuration file (“package.json”) for the Node.js eco-system is required and no additional boiler-plate code is needed. A "hello world" application can be written in ~10 lines of code.

const fastify = require('fastify')({ logger: true })
fastify.get('/', function (request, reply) {
    reply.send({hello: 'world'})
})
fastify.listen({port: 3000}, function (err, address) {
    if (err) {
        fastify.log.error(err)
        process.exit(1)
    }
})

"Hello World" in Fastify

Pros

• Plugin architecture can accelerate development

• Plugins can easily be reused in different projects

• Fastify supports TypeScript (declaration file is maintained)

• Fastify can serve up to 30k requests per second (claims by Fastify)

• The plugin system allows an easy shift from monolithic applications to microservices (when the context is configured correctly)

• Good official documentation resources

Cons

• Understanding the application context and scope of the plugin registration can be complicated at first

• Low overhead also means that all required functionality has to be self-implemented

• Files can become very verbose (especially routes with parameter definitions)

• Relatively small development community (few online discussions/threads)

• No TypeScript documentation is available, only JavaScript

Lessons Learned

Picking the right tech stack is one of the most important decisions when trying to accelerate the software development process. The best choice varies from project to project and it can be difficult to find the sweet spot between high and low/no-code solutions. High-code frameworks require lots of setup and configuration time but come with a lot of functionality out of the box (e.g. Spring Boot). Low-code solutions provide pre-built components that enable developers to bootstrap an application in minutes but might not be as flexible when custom functionality is required. Depending on the chosen technologies and frameworks a trade-off between flexibility and invested time always has to be made when considering development overhead.

The Fastify framework has very low overhead, but this also means that every required functionality needs to be self-implemented or at least self-configured if there is an existing plugin. The plugin architecture is a great system to accelerate development, especially if there is a baseline of existing plugins from previous projects. If a team is confident in working with it, Fastify can be a great framework choice.

React-Admin provides useful abstractions to quickly build simple CRUD-based dashboard-style apps in a few lines of code. If developers are working within the constraints of React-Admin, satisfactory results can be achieved rapidly, but implementing custom features can be very restrictive when working with the framework. Depending on the project scope, React-Admin can be a solid choice. It is best used for less complex projects with resource inspection and manipulation as the main focus.

Sadly there is no clear-cut solution to increase development speed by picking the right frameworks. All project requirements (and possible requirements in the future) need to be considered to make an educated decision. In the end, frameworks are just tools that should help developers fulfil the requirements, but suboptimal framework choices can negatively impact development time or even lead to project failure. The best framework choice is often the one teams are most experienced with because key concepts and limitations are known beforehand. When unfamiliar frameworks are proposed to be used, time should be spent on research and preparation to avoid problems in the future. If the correct framework is chosen for a specific use case, development speed can be increased significantly.

We hope that you learned something by reading this article and maybe gained a new perspective on framework choice. React-Admin and Fastify can be solid options and hopefully, you received some insight on whether they might be a good fit for one of your future projects. Keep on coding, cheers!

In today's data-driven world, organizations big and small rely on data to make informed decisions, gain insights, and drive business growth. However, raw data alone is seldom enough; it needs to be transformed into actionable information. This is where Power BI steps in, offering a powerful suite of tools to analyze, visualize, and share data.

Before looking into the possibilities and techniques on how to embed Power BI reports - which is one of the most important questions in our custom software projects, let's have a brief look at the question:

What is Power BI and which key features does it offer?

In brief, Power BI is a business intelligence and data visualization tool that enables users to turn raw data into interactive visual reports and dashboards. It provides a unified platform for data exploration, data preparation, data modeling, and data sharing. With its intuitive interface and robust capabilities, Power BI has become the go-to choice for organizations seeking to extract meaningful insights from their data.

To give an idea, let's look at a sample report (which can be downloaded here):

Key Features

1. Data Integration

Power BI connects to a wide range of data sources, including databases, cloud services, and on-premises data. It can seamlessly integrate data from Excel spreadsheets, SQL databases, DataVerse, Azure, and many more, making it easy to consolidate and transform data from various sources into a single dataset.

2. Data Transformation and Modeling

Power BI offers a powerful data modeling engine that allows users to shape and transform data using the Power Query Editor. This tool is particularly useful for cleaning, filtering, and structuring data before analysis. Users can create relationships between tables, define calculated columns and measures, and apply advanced transformations.

3. Interactive Visualization

One of Power BI's standout features is its ability to create stunning visualizations. Users can choose from a wide range of charts, graphs, and maps to display data in a compelling and informative way. The drag-and-drop interface makes it easy to create interactive dashboards that update in real-time as data changes.

4. Collaboration and Sharing

Power BI enables collaboration by allowing users to share reports and dashboards with colleagues or external stakeholders. The ability to share reports and collaborate on data analysis fosters a culture of collaboration within organizations, promoting better knowledge sharing.

5. AI and Machine Learning Integration

Power BI integrates with Azure Machine Learning, allowing users to embed machine learning models into their reports and dashboards. This enables predictive analytics, anomaly detection, automated insights generation, and a Q&A feature. It uses natural language processing to generate visualizations and answers based on the data available.

6. Mobile Access

With Power BI Mobile, users can access their reports and dashboards on smartphones and tablets. This ensures that decision-makers have access to critical data wherever they are.

Licenses

Choosing the right Power BI license is crucial for maximizing the benefits of this powerful tool while staying within a defined budget.

In general, licensing can be split into two main categories, user-based (Free, Pro, Premium Per User) and capacity-based (Premium, Embedded) licensing.

Integrating Power BI Reports

If you are already developing applications or webpages based on the Power Platform ecosystem (CLF Engineering Blog) then embedding a report into Power Apps is your way to go.

Integrating into Power Apps or Power Pages

This integration allows you to embed Power BI content directly into web pages hosted on a Power Apps Portal, providing a seamless user experience for external customers, or users who may not have direct access to Power BI. Within your Power Apps Portal, you can embed Power BI reports or dashboards into web pages. This can be done using the "Power BI" component or HTML iframe.

For more details, you can refer to the official documentation or our Cloudflight Engineering Blog.

Integrating into a Custom Application

For us - as a company developing custom software - the most interesting approach is to embed reports into other applications using Power BI Embedded.

To ensure that only authorized users can view embedded Power BI content, a robust authentication and authorization system is required and that is where Power BI Embedded Tokens come into play.

Power BI Embedded tokens are a type of security token that grants access to specific Power BI content. They are used to authenticate users and control their access to embedded reports and dashboards. Here's how they work:

• Generate Token: When a user requests to view an embedded Power BI report or dashboard, the hosting application (our custom-developed applications) needs to authenticate the user with Power BI. It does this by requesting an embedded token.

• Token Parameters: The token request typically includes parameters such as the user's identity and roles and the specific report/dashboard to be accessed. These parameters determine what the user is allowed to see.

• Token Issuance: Power BI generates a token based on the provided parameters. This token is a temporary, time-limited access key.

• Access Control: The token contains information about the user's permissions, roles and the content they are allowed to access. When the user tries to access the embedded content, the token is validated to ensure the user has the necessary permissions.

• Expiration: Power BI Embedded tokens have a limited lifespan, which enhances security. Once the token expires, the backend must request a new one for continued access.

Let's have a look at the steps we implemented to acquire embedded tokens:

1. User Authentication: Firstly, the user of your web application goes through an authentication process within your web app using your chosen authentication method. This step verifies the user's identity.

2. Web App Authorization: Our web application, having successfully authenticated the user, utilizes a service principal to establish authentication with Azure Active Directory (Azure AD). This step grants our web app the necessary permissions for interaction with Power BI REST APIs by requesting an Azure AD token.

3. Embed Token Request: Our web application communicates with the Power BI Embed Token REST API operation, initiating a request for an embed token. This specific token defines precisely which Power BI content can be embedded within our application as explained above. In response to our request, the REST API provides our web application with the embed token, which is specific to the requested Power BI content.

4. Passing the Embed Token: Our web application then securely passes this embed token to the user's web browser, allowing the user's browser to facilitate the interaction with Power BI.

5. User Access: Finally, the web app user employs the embed token within their browser to access and interact with Power BI content, as authorized by the token's permissions.

Feel free to have a look at the source code provided by Microsoft to see how you can acquire an Azure AD token using a service principal and how to generate embed tokens.

Regarding the frontend integration, Power BI supports all major UI frameworks - Angular, VueJS and React.

<powerbi-report
    [embedConfig] = {{
        type: "report",
        id: "<Report Id>",
        embedUrl: "<Embed Url>",
        accessToken: "<Access Token>",
        tokenType: models.TokenType.Embed,
        settings: {
            panes: {
                filters: {
                    expanded: false,
                    visible: false
                }
            },
            background: models.BackgroundType.Transparent,
        }
    }}

    [cssClassName] = { "reportClass" }

    [phasedEmbedding] = { false }

    [eventHandlers] = {
        new Map([
            ['loaded', () => console.log('Report loaded');],
            ['rendered', () => console.log('Report rendered');],
            ['error', (event) => console.log(event.detail);]
        ])
    }
>
</powerbi-report>

Row Level Security

Row Level Security (RLS) in Power BI is a security feature that allows you to control access to data at the row level based on user roles and filters. This means you can restrict what data individual users or groups of users can see within a Power BI report or dataset.

• RLS is typically implemented by creating user roles within your Power BI model. Each user role can have specific data access rules associated with it. User roles can be defined and managed in Power BI Desktop or through Power BI service.

• To enforce RLS, you use filter expressions within user roles. These filter expressions are written in a DAX (Data Analysis Expressions) language and define which rows of data are visible to users in that role. Filter expressions can be as simple or complex as needed, allowing you to create dynamic filters based on user attributes, such as username or department. You can find an example usage of DAX expressions where the users will see their data rows ONLY if the underlying data set has a record under their username.

• RLS provides dynamic security, meaning that the data is filtered in real time as users interact with the report or dataset. Users will only see the data that aligns with their role and the applied filters. The filters can be applied to specific tables and roles as follows:

• Once RLS rules are defined and tested, you can publish your Power BI report or dataset to the Power BI service. RLS rules are enforced in the service as well, ensuring consistent security across different platforms and devices.

Example of using RLS for filtering per person:

We want to filter everything in the sample report such that the logged-in executive can only see their data. For that, we will have to use USERPRINCIPALNAME() in DAX which will return the logged-in executive. Create a new measure on the table that has the executive names and name it "User" with the value USERPRINCIPALNAME(). Then, create the security role by clicking on the Modeling tab -> Manage Roles. Create a role and then define the filter. This filter simply means that the logged-in user will only see his/her records in the whole data set:

This is how the sample report looks when we view it as the executive "Andrew Ma" after RLS:

Example of using RLS to hide/show items depending on the security role:

A function to hide an entire page or specific elements in a report for certain roles is not available but there is a workaround on PowerBI Desktop. You can build your report as normal, then add a card and make it big, so that it overlays all the things you want to hide.

Warning: Note that this workaround would only visually hide the report from the UI and a malicious user would still have access to the data in the underlying dataset. Therefore, it should not solely be employed in combination with RLS when working with real data.

To do that, enter new data and create the following table:

Create a new measure and write the following DAX which creates the message you want to display:

• Message = IF(HASONEFILTER('RLS Table'[RLS]), "You are not authorized!", "")

Create another measure and write the following DAX which controls the background of the overlay card we will create:

• Make Transparent = IF(HASONEFILTER('RLS Table'[RLS]), "#White", "#FFFFFF00")

Create a card and make it as big as the report page. Choose the "Message" measure to be displayed on it.

Format the RLS Table's visual in the general tab and find the background properties. Set the first dropdown to "Field value" and the second to "Make Transparent".

Create a new role that will not be authorized to see the report:

This is how the report should look like when you make the overlay card as big as the report:

Conclusion

Power BI is a powerful tool that empowers organizations to harness the full potential of their data. With the possibilities for embedding reports into applications, it is a valuable puzzle piece in our toolkit when it comes to efficiently implementing custom software solutions.

At first glance sending an email is just a line of code, right? Well, integrating any asynchronous messaging functionality (e.g. sending emails, sending data to 3rd party services, billing systems etc.) into software applications can be a daunting task, especially when it comes to ensuring the reliability and consistency of message delivery. Most of the time applications need a way to send one message if and only if the database gets updated with an entry. A simple example of when this behavior is necessary would be user registration. When a new user registers, the application has to send a confirmation email to their address. Seems easy enough, so what could go wrong?

Let us try writing some Spring Boot code with Kotlin to illustrate the problem:

@Transactional
fun registerUser(user: User){
    userRepo.save(user)
    emailService.send(ConfirmationEmail())
}

At first sight, this code may seem right. But what happens if the server encounters an error after saving the new user? The email will get sent but the user will not exist in the database. Now, you might think of wrapping this code in a try-catch block such that we don't execute the send function if the save operation fails. This would look something like this:

@Transactional
fun registerUser(user: User){
    try {
        userRepo.save(user)
        emailService.send(ConfirmationEmail())
    }
    catch(err: UserRepoException) {
        // ...    
    }
}

Unfortunately, this isn't a good approach either. What if the email provider is not available at the moment? The user will be persisted in the database, but the confirmation email will never arrive to them. It will not even be sent to begin with. This seems unacceptable in a professional, modern web application. We would like to somehow roll back the saving of the user if sending of the email fails. This might remind you of the lesson about transactions from the databases course.

Transactional Outbox Pattern to the Rescue

The Transactional Outbox Pattern, a proven architectural design, provides a robust and reliable solution for managing email sending within applications. It addresses various use cases where message delivery is crucial and helps mitigate potential failures that can occur when this pattern is not implemented.

The basic idea is to have an Outbox table that contains the emails our application has to send out. We can now insert, in the same transaction, into this new table and the User table when a new user is created.

Key Components of the Transactional Outbox Pattern:

1. Outbox: The central component of this pattern is the "outbox," which acts as a temporary storage for messages that need to be sent. When a message needs to be sent (e.g. an email), instead of sending it directly, it is first placed in the outbox, in the same transaction as the update on the other table (e.g. Insert in the User table). This outbox can be implemented as a database table, a message queue, or any other persistent storage. In our case, it will be a database table.

2. Message Queue or Scheduler: An essential part of the pattern is a mechanism that monitors the outbox for messages and sends them at an appropriate time. This can be done using a message queue (e.g. RabbitMQ, Kafka) or a scheduler that periodically checks the outbox for pending messages. When a message is sent successfully, it is marked as "sent" in the outbox.

3. Transactional Behavior: The Transactional Outbox Pattern ensures that message sending is part of a larger transaction as an atomic database operation. If the transaction fails (e.g., due to an error or an exception), the messages are not inserted into the outbox and the data is not updated in the other table (UsersTable in our example). This guarantees that messages are only sent when the whole transaction is successful, maintaining data consistency. This behavior can be easily achieved in Spring JPA using the @Transactional annotation since it satisfies the "ACID" requirements: it is Atomic, Consistent, Isolated and Durable.

Sequence Diagram

We can illustrate the flow of registering a new user in an application that implements the Transactional Outbox pattern. It would look something like this:

Implementation

Now that we understand the problem and have a solution, we can write an EmailService class to achieve reliable email sending:

@Service
class TransactionalEmailService(
    val outboxRepository: OutboxRepository,
    val emailOutboxMapper: EmailOutboxMapper
) {
    @Transactional
    override fun sendEmail(email: Email) {
        this.outboxRepository.save(this.emailOutboxMapper.emailToOutbox(email))
    }
}

The service has only one function: sendEmail(). The function saves the Email object into the Outbox table as a database entry. It is annotated with @Transactional, allowing Spring to handle the database save operation as part of a transaction. Please note that the default propagation for @Transactional is required since we want the code in the sendEmail() function to run in the same transaction as the code in the function calling it (so both the original database operation and the save on the outbox repository happen in the same transaction). Using another propagation might result in creating a second transaction for sendEmail(), which would defeat the purpose. For more information please check the official documentation.

Scheduled task (Email Relay)

To fetch the Outbox table for new entries we decided to use a scheduled task. This task fetches batches (pages) of entries from the Outbox until there aren't any un-fetched entries. It is important to implement this way and to not only fetch one batch per execution since, if at any point a lot of messages enter the Outbox (if a newsletter has to be sent, for example), the scheduled task will not stop until it has tried to send every message, thus saving the time it would have had to wait between the scheduled runs.

We should also mention that in the Outbox table, besides the usual email fields, we also store:

• the scheduled date to send the email, which serves two purposes: to send emails at a certain point in the future; and if sending fails, mark at which point a resend should be retried

• number of tries: if an email fails too many times the service will stop trying to send it

To optimize our selects, we created a compound index on the Outbox table on the columns ID, Number of tries and Scheduled date.

A pitfall we have to watch out for is that we may have multiple instances of the scheduled task running at once. To make sure that the two tasks don't select the same batch of emails (which would result in sending the same emails multiple times) we can use a distributed lock, like ShedLock. Its purpose is to make sure only one instance of the scheduled task is running at a certain point in time. Another way, which is a bit more database management system specific is using the row locks FOR UPDATE SKIP LOCKED. This command is available for PostgreSQL, Oracle and others but not present for example in SQL Server and SQLite.

A pseudocode of our implementation would look something like this:

acquire distributed lock
repeat until no batches left {
    batch = select relevant entries from Outbox table
    for each email in batch {
        try {
            send(email)
            // send - successful
            delete email from Outbox table
        }
        catch {
            // send - failed
            email -> increase number of tries
            email -> set scheduled send date sometime in future   // (time of retry)
            update email in Outbox table       
        }
    }
}
release distributed lock

Email Sender

There are multiple approaches to handling this step, and it is very project-specific. You might use an SMTP server or something like AWS SES. At this point, there isn't anything that can go wrong as long as you are careful to catch all exceptions in the Scheduled Task so that they are handled properly.

We created a custom AWS SES Sender class that implements JavaMailSender. This way we can easily switch between the default JavaMailSenderImpl SMTP implementation and AWS SES implementation just by changing a configuration. This is Spring Boot specific, but it should be pretty similar in other languages and frameworks. Our implementation looks something like this:

@ConditionalOnProperty(
    value = ["email.sender"],
    havingValue = "AWS",
    matchIfMissing = false
)
@Component
class AWSSESJavaMailSender(
    @Autowired val sesClient: AmazonSimpleEmailService
): JavaMailSender {

    override fun send(mimeMessage: MimeMessage) {
        try {
            val outputStream = ByteArrayOutputStream()
            mimeMessage.writeTo(outputStream)

            val buf = ByteBuffer.wrap(outputStream.toByteArray())

            val rawMessage = RawMessage(buf)

            val rawEmailRequest = SendRawEmailRequest(rawMessage)

            sesClient.sendRawEmail(rawEmailRequest)
        }
        catch (ex: Exception) {
            throw MailSendException("Could not send email through AWS SES", ex)
        }
    }

    override fun createMimeMessage(): MimeMessage {
        return MimeMessage(Session.getDefaultInstance(Properties()))
    }

    // other functions' from  JavaMailSender implementation...
}

Usage

The purpose of this project was to implement the pattern in an easily reusable way for our Spring Boot projects. We provided a way to make the pattern easy to use while keeping the code clean.

The sendEmail() function of the TransactionalEmailService can be called in any other transactional function:

@Service
class ExampleEmailServiceImpl(
    val userRepo: UserRepo,
    val emailService: TransactionalEmailService
){
    @Transactional
    fun exampleUsage() {
        this.userRepo.save(User(...))
        val email = Email(...)
        this.emailService.sendEmail(email)
  }
}

Note that both the exampleUsage() function and our sendEmail() function are annotated with @Transactional. Spring is smart enough to handle both database changes in a single transaction, fulfilling the ACID requirements.

For a more decoupled approach we can make use of event listeners:

@Service
class ExampleUserService(
    val userRepo: UserRepo,
    val eventPublisher: ApplicationEventPublisher
){
    @Transactional
    fun exampleUsage() {
        val user = userRepo.save(User(...))
        eventPublisher.publish(UserCreatedEvent(user))
    }
}

@Service
class UserEventListener(
    val emailService: TransactionalEmailService
) {
    @Transactional
    @EventListener
    override fun handleUserCreatedEvent(event: UserCreatedEvent) {
        val email = Email(...)
        emailService.sendEmail(email);
    }
}

Conclusion

If right now you're thinking about that project that you worked on in the past and you didn't implement information sending in a truly reliable way, you're not alone. This is a very common mistake unfortunately, but we hope that through this article we were able to paint a clearer picture about this design pattern, why it is useful and how to implement it.

While our roots are firmly planted in traditional software development methodologies, our commitment to innovation and client satisfaction drives us to continuously explore new horizons. This leads us to consider various emerging platforms, like Microsoft Power Platform, as potential additions to our already extensive toolkit.
While this might seem a bit unconventional for a software development company, our motivation is rooted in a deeper understanding of the ever-evolving business landscape and the diverse needs of our clients.

"What was your goal?", you ask.

In brief:

• Fast results

  How much faster can we implement solutions when working in an environment with topics such as authentication, infrastructure and deployment, more or less out of the box?

• Customer empowerment

  Can the use of something like Power Platform help clients maintain their applications, run updates, and introduce new features with little to no prior coding knowledge?

• Project diversity

  Which projects do we see this approach as being beneficial, optimal even?
  Are there any blockers that could prevent us from following this approach?

Overview

Power Platform Architecture

Microsoft Power Platform is a suite of integrated tools and services designed to assist individuals and organizations in creating custom business applications, automating workflows, analyzing data and generating reports.
It's comprised of four core components:

1. Power Apps & Power Pages
   They allow users to create custom applications without extensive coding knowledge, to begin with. They offer canvas apps and model-driven apps: canvas apps enable building applications with a visual interface, while model-driven apps are more data-centric and built on the Common Data Service. Both interact with (business) data stored in the Dataverse with the main difference being that Power Pages is targeted at external users backed by the many authentication options (e.g. Azure AD, LinkedIn, Facebook) that are available out of the box.

2. Power Automate
   Formerly known as Microsoft Flow, enables the automation of repetitive tasks and workflows across various applications and services. It connects to hundreds of apps and services, allowing users to create automated processes without much complexity (e.g. automated email sending).

3. Power BI
   A powerful business analytics tool that allows users to visualize data, generate reports and build dynamic dashboards. It's particularly useful for data analysis and decision-making by turning raw data into actionable insights.

4. Power Virtual Agents
   Allows users to create chatbots that can be used to engage with customers, provide support, answer queries and automate various interactions.

Aiming to fulfill the requirements mentioned previously, we ultimately focused on the use of Power Pages and Power Automate.

Pros & Cons

Next, we'll share some useful insights gained along the way.

Pricing & Costs

First, a word of advice - make sure to understand the licensing and billing of the entire Power Platform ecosystem to avoid unpleasant and costly surprises. In our setup, the main cost driver (besides Dataverse storage) was the amount of monthly active users, both anonymous and authenticated. Prices vary depending on the chosen subscription plan, meaning either a package of a set number of users or the option of "pay as you go" (PAYG). Technically, we used a billing policy linked directly to an active Azure subscription.
That being said: be aware that current costs are not immediately visible and it can take up to 24 hours until they become available within Azure cost management.

"Why is there so much emphasis on pricing?", you might be asking.

Imagine a scenario where someone sets up a basic cron job that operates on the Dataverse every minute or so. A premium flow that can either run in the cloud or attended costs $0.60 per run, mind you.

We'll let you do the math and figure out how high the costs get when the cost management alerts start triggering, which - as explained above - can take up to 24 hours to be updated.

Spoiler alert: It's $864.

Use a Power Automate license in similar scenarios.

Speaking of monthly active users...

They are reported in a daily summary, downloadable within the Power Admin Portal.
Considering the uniqueness of an anonymous user being tracked using a browser cookie, we see a potential risk of unpredictable costs as a consequence. The claim is that malicious attacks, bots and crawlers are excluded from being counted, but we could not 100% confirm this statement.

TL;DR

• Understanding the main use case for the application, with regards to target users, is highly advised to avoid precarious financial situations

• Make sure to choose the optimal licensing model from the very beginning

• As in all cloud projects: enable cost alerts from Day One!

Development

As with every new technology, there will be a learning curve. It's not particularly steep when it comes to the Power Platform itself, but one has to be wary that the workflow is not comparable to a 'traditional' software development process. Coupled with the documentation provided by Microsoft, which sadly tends to be a bit outdated at times, this can be a cause for sudden spikes in the overall learning curve.

Coming into this methodology, it took us some time to get used to the way collaborative work can be set up on a shared Power Pages environment. Because there's no implementation of version control, merge requests or individual commits, it took further coordination to avoid interfering with one another. This is easily manageable for smaller-scale projects, though not recommended for larger projects. In turn, we recommend a team size of no more than 3 people for optimal workflow.

One undisputable upside is that coming from a background of data-driven applications with CRUDL (Create-Read-Update-Delete-List), overall development time is crazy fast. It essentially boils down to "creating tables, views and forms (rinse and repeat)". Development is further sped up by the use of provided components for features, such as:

• Login

• User Registration

• Authentication Providers

TL;DR

• Ideal for small, low-complexity projects

• Works best for smalled-sized teams (2-3 people max)

• Close to no collaborative features, a developer would normally be accustomed to

• Always use (and re-use) components whenever possible

Testing

We know how much developers love testing code (*sarcasm*). Good news: manual testing is pretty much your main 'go-to' approach in the Power Platform. For those interested in automated testing, we can use our default framework (Cypress), triggered by an external build.

Deployment

When it comes to deployments, we set up a pipeline that can be executed within the Power Platform, together with the typical pattern of cross-environment deployment: Dev → Test → Prod.
We have noticed that it is sufficient for the before-mentioned, smaller projects.
Of course, it is not as powerful as either Gitlab or TeamCity, but that would not be needed.

Scale, Expand & Maintain...ability

In terms of scalability, besides maybe investing more money into a broader user base or file storage capabilities, not much to add in this regard. If the final goal is to create smaller, lower complexity applications, we think the expandability options provided should suffice (an API for CRUDL can be used, as well as Azure Functions with an HTTP trigger, for more complex logic).

Similarly, with the premise of a smaller application, maintainability options offered should again suffice for most projects.

i18n ("Internationalization")

i18 support is provided on the platform. There's a reasonable list of available languages that can be activated for every given page, the pattern used is a 'key-value'-type approach: language → translation.
Except for a few components that are translated automatically (i.e. login), translations must be provided contextually on each page, for each desired language. On the flip side, there's the option of providing a resource file for said translations (that being said, we did not delve into this approach enough to confirm its utility).

Lessons Learned

For some, Microsoft's Power Platform might seem to only thrive in specific scenarios (think one-trick pony). We believe that it has earned its place in the ecosystem of modern technology stacks, and when it comes to:

• Rapid Fire MVPs

• Purely data-driven projects

• "Set it and forget it"-type projects

• Pre-defined user bases

• Minimal UI

• Pre-existing Microsoft ecosystem integration

• "Developing without developers" (Welcome to 2023)

there is close to no competition. We say "give it a try".
It might end up being the solution you were looking for.

As programmers, we often revel in the most technical-complete solutions and prefer writing our services from the ground up. That is both a blessing and a curse: the control, efficiency, and power of writing everything yourself comes at the cost of speed in development. As part of our Cloudflight technical lab, we decided to explore different solutions that help in delivering software faster - rapid development. This is specifically useful when our teams are asked to deliver a functional MVP in a very short time.

If you want to learn more about the challenge please read the introduction article first.

Tech stack

Strapi

Strapi is the biggest selling point of our solution. It is a powerful, open-source Node.js and TypeScript Content Management System (CMS). At its core, it is a shortcut for creating REST or GraphQL APIs, replacing the need to write all the backend code yourself. The CMS part, Content Management System, basically means that Strapi provides an admin dashboard running on a web application where you can view your database data, create new API endpoints, and build your server using an intuitive GUI.

Strapi is the leading open-source headless CMS. It’s 100% Javascript, fully customizable, and developer-first. Strapi

Diving deeper inside the inner workings of Strapi, we found out that it uses Koa as a web framework and Bookshelf.js, which is powered by Knex, as an Object Relational Mapping (ORM). On top of these two Node.js libraries a "Strapi framework" was built, which can be used by the autogenerated code, or programmatically in case you don't want or can't use the dashboard generator for a specific case. This framework provides generic Controllers, Services to handle CRUD requests, and a generic Entity Service to handle database operations, like find, create, update, and delete. This makes it very easy to customize Strapi, and also for the dashboard to generate the code.

Some nice-to-have feature of the generated code is the possibility to add complex filtering and join operations right in the requests. The format of the parameters gets converted by the Entity Service into a Knex ORM query, which then translates into SQL. Another great thing is out-of-the-box pagination.

Customizing Strapi is generally very facile. Most of the time the only thing that the developers need to do is add some extra functions, inside an object, passed as a parameter to the constructors of the Controllers or Services. You can use them for custom validations or business logic, and they have access to the original Controllers generated for that entity, and all the Services and Entity Service, so developers don't have to reinvent the wheel. For more complex use cases custom endpoints and database queries can be written.

import {factories} from '@strapi/strapi';
export default factories.createCoreController('api::reservation.reservation', 
({strapi}) => ({
    // ...
    async find(ctx) {
        const {data, meta} = await super.find(ctx);
        // validation for admin users
        if (ctx.state.user.role.name === 'Admin') {
            return {data, meta};
        }

        // custom validation for regular users
        const userId: number = ctx.state.user.id;
        if (data.some((el) => el.attributes.users_permissions_user.data.id !== 
userId)) {
            return ctx.forbidden('Data not created by this user', {});
        }
        return {data, meta};
    },
    // ...
}));

This code is essentially creating a controller for a reservation API endpoint, with special handling for admin users and regular users. Admins can access reservation data without restrictions, while regular users can only access data that they have created. If they try to access data created by other users, they receive a "forbidden" response.

None of us had any experience working with Strapi or anything remotely similar before. As such, we used pair programming in the first week of working together. It proved to be an excellent use case of our time, as we all collaborated in understanding how to work with Strapi. Once we all got used to the overall architecture of our app, we only pair programmed on the important stuff. In the end, we believe it is very easy and intuitive to use, as a couple of days of research were enough to use Strapi to its full potential, as a self-sufficient complex backend service.

Plugins

The true strength of Strapi lies in its plugins, which can be added free of charge from Strapi's market (built into the admin dashboard). They add additional functionalities to Strapi, extending its capabilities with no additional code. Some of the most useful plugins we have discovered:

• Documentation: generates an OpenAPI document for all of Strapi's endpoints that can either be opened on the browser or be used to generate code down the line. Add this plugin to list all available endpoints and see how to properly make requests to Strapi.

• Config Sync: allows programmers to share Strapi settings between environments, like access rights to different operations based on roles, either from the CLI or the GUI. This plugin is a must if multiple people are working on the project, or if you want to deploy Strapi.

Other plugins that were preinstalled in our initial Strapi project:

• Content Type Builder: Add new data tables in your database from the Strapi GUI. This is what allows you to build new entities and CRUDs. In Strapi, entities can be Collection Types or Single Types. The difference between them is that Single Types only allow for a single value, acting like singletons, while Collection Types are regular tables that support multiple rows.

• Content Manager: Quick, code-less way to see, edit, and delete the data in your database. This may also allow the owner of the application to manage content without redeployment.

• Email: Configures the application to send emails. This plugin helps you format your emails and send them further to 3rd party providers.

• Media Library: Load images and use them in your API - you can easily store multiple image types and use them on your website.

• Roles & Permissions: JWT-based API security and user management system. This adds the User data type in your application, which you can use for authentication purposes. It supports multiple providers and multiple security roles.

• Internationalization: Adds the ability to create new locales and set up i18n for your API. Having this plugin allows the owner of the app to localize content without redeployment. The API will return the content for the right locale based on your API request parameters.

Performance

We load-tested Strapi's create reservation endpoint using Apache JMeter to have a reliable, reproducible result. We used a variable number of users, increasing it over time. We haven't experienced any performance issues so far.

Mail Sender

Strapi has a built-in email plugin, but it doesn't automatically send the emails, it just formats them and forwards them to certain providers. There are some providers (node modules) built in, but they are either 3rd party platforms, or they are bare bones (such as nodemailer) so, if you want to handle failures yourself and retry after some time, or you don't want an external service to process your emails, you need to create your own provider. Providers are node modules that extend the functionality of a Strapi plugin.

Since we decided to implement email sending using RabbitMQ and a microservice, our provider pushes the formatted email that it receives from the Strapi email plugin to the specific RabbitMQ exchange. Then, this exchange sends it to an email queue, where it will be read by our email processing microservice.

To reliably send emails we decided to set up RabbitMQ so that we have an email queue, a retry queue, and an error queue. In the email queue, we have the emails that need to be processed by our microservice. If, when trying to send the email an error occurs, the email sender will post the message inside the retry queue if the number of trials to send the specific message is smaller than a set constant. Otherwise, it is going to send it to the error queue. In the retry queue, emails remain for a specific amount of time (they have a time to live), then they get discarded and sent back to the email queue.

This approach guarantees that in most cases our service is going to be able to send the emails, and if not, they will remain in the error queue.

VueJS

All webpages are implemented in VueJS using Composition API. Using our previous experience in Angular and React made VueJS easy to pick up. The official documentation and tutorial were very helpful. To speed up development, we have implemented generic components for all views of our app: a generic form using Vuelidate for validation, a generic table with edit and delete actions, and a generic object details card. Using Pinia, it was also very easy to implement reactive stores.

We decided to use the free version of CoreUI as the main components library. The date picker component of CoreUI is locked behind a paywall, and we struggled at first to find a reliable date picker component for our reservations view. We settled on using the date picker of PrimeVue.

OpenAPI generator

Most web developers have seen the OpenAPI specification before, as it is what Swagger tools use. Naturally, we decided to create this specification for Strapi. Upon adding the Documentation plugin to our project, we gained access to a .json file containing the full list of our endpoints in OpenAPI specification. This .json file is generated every time we start the Strapi development server. As such, we can open this file using Swagger to view and better understand how to use the APIs provided by Strapi. This plugin allows for customization of the resulting specification file, like the ability to include or exclude properties or endpoints.

Ultimately, we used the OpenAPI specification of our API to also automatically generate REST services for the frontend that can communicate with Strapi. The tool we used generates Typescript files which we add to our VueJS project. This was the biggest timesaver in our frontend development. Usually, writing the frontend REST services yourself is a redundant task that should be automatized based on your API of choice's specifications.

Postgres

PostgreSQL is the official database system recommendation in the setup guide, but multiple database engines are supported. We have used Postgres due to personal preference and convenience.

Spring boot - where is it?

The initial tech stack included a self-managed Spring boot application, acting as a man in the middle between the VueJS frontend and the Strapi backend. We removed this layer of complexity in the final solution because Strapi was sufficient enough to secure endpoints and capable enough of adding our app's business logic. Thus, adding Spring would have been redundant and would have just added extra complexity, another possible bottleneck, and would have taken more time to develop. Of course, for larger applications there might be requirements that Strapi might not be suitable for, so a Spring backend might be required as well.

Pros/cons discussions

This is an overview of using Strapi for our backend services:

Pros

• Facile implementation of CRUDs for models with simple logic

• Easy to use: using the GUI for boilerplate code is very convenient

• Free to use for any project, if managed on your own premises

• Can be extended with custom implementations (you're not limited by the auto-generated code)

• Comes with multiple authentication providers out of the box

• Offers a lot of plugins, both official and community-made

• Multiple SQL database types are supported

• Applications may be both vertically and horizontally scaled

Cons

• Less time-efficient than a self-managed service

• Lack of documentation - debugging can be slow

• Unexpected crashes for the CMS /admin dashboard while editing the schemas of the collections (not a problem for production - modifying the schemas is disabled in production)

• Incomplete migrations support - migrations run before schemas are updated since schemas are managed by Strapi and migrations by Knex

• Virtually no validations support - need to do custom endpoints

• Does not support MongoDB natively

Lessons Learned

There is no tech stack right for every project, but for sure there is a best tool for the job. In our case, considering the functional requirements and the deadline, Strapi did a great job. Using a monolithic architecture and built-in Strapi features was the fastest way for us to deliver the MVP in the given period. We found out about Strapi that it is a very powerful and versatile way of building backend services and using it was a satisfying experience. It is also both database and frontend agnostic. We definitely will consider it in the future for new projects, either as a stand-alone, self-sufficient server or as a microservice, since it is more than suitable for delivering MVPs or for creating a reliable microservice in larger projects. It is capable of reducing development time and it is quite easy to pick up. However, for the best experience, we recommend installing the plugins of your Strapi project immediately upon setup, to prevent possible merge conflicts and other issues during collaboration with your team members.

Please note that there are also small caveats, like the one that we learned about when we stumbled upon difficulties as we tried to introduce complex logic and validations. For instance, we had 2 entities with a one-to-many relationship between them, but found out that Strapi did not support cascade delete in the default API implementation (see feature request). As such, we had to add a database-level trigger to delete the ends of the one-to-many relationship.

Another lesson we learned is that Strapi can scale. If you want to scale your application horizontally, you may use a load balancer like Nginx. For additional optimization, you will want to implement caching between sessions. This can be achieved by using the Rest Cache Strapi plugin with a self-managed Redis server.

Also, we find VueJS to be a technology worth using in the frontend frameworks ecosystem. Previous experience with other component-based frameworks is a plus, but the official documentation and forums are a great way to start. As an event-driven architecture, it is intuitive to learn and use.

Let's roll the credits (in alphabetical order) and wrap up this chapter in our software dev diary: Andrei Cotor, Daniel Todașcă, and Gergely-Péter Mátyás contributed to this exploration. Remember, the code may compile, but the journey is what truly matters. Stay curious, stay coding, and let's keep pushing the boundaries of what's possible, one line of code at a time. Until next time, happy coding, my fellow developers!