Skip to main content

#3690 Language Requirements (approved)

About this Document​

This RFC lists the requirements from the wing language experience, not the specific syntax/grammer or design of the language. The language design will follow in a separate RFC.

Each section of this document has a requirement tag that can be used to reference it in future documents, roadmaps, github issues, etc. The doc also includes an HTML anchor for each tag (e.g. see #w:program-output)

For example:

Reqtag: w:my-tag

Overview​

The wing programming language (aka winglang) is a general-purpose programming language designed for building applications for the cloud.

What makes wing special? Traditional programming languages are designed around the premise of telling a single machine what to do. The output of the compiler is a program that can be executed on that machine. But cloud applications are distributed systems that consist of code running across multiple machines and which intimately use various cloud resources and services to achieve their business goals.

wing’s goal is to allow developers to express all pieces of a cloud application using the same programming language. This way, we can leverage the power of the compiler to deeply understand the intent of the developer and implement them through the mechanics of the cloud.

Programs​

When a wing program is compiled, the output is not an executable which runs on a single machine, it is a set of files which are synthesized during compilation by your wing code.

Reqtag: w:program-output

At the most basic level, the wing compiler can synthesize any file and directory structure. When used for creating cloud applications, these files are a set of infrastructure definitions (such as CloudFormation, Terraform or Kubernetes manifests), Dockerfiles, function code bundles, deployment workflows, and any other artifact that is needed in order to deliver this application to the cloud. wing is not opinionated about cloud providers and designed to support cloud applications running on Azure, GCP, AWS and any other provider. It is also not opinionated provisioning engines and can support Terraform, AWS CloudFormation, Pulumi, and any other format.

Let's look at a very basic example:

bring std;

let hello = std.TextFile("hello.txt");
hello.add_line("Hello, world!");

Now, let's compile this program:

$ wingc hello.w
$ cat hello.txt
Hello, world!

Now, let's update our code:

let hello = std.TextFile("hello2.txt");
hello.add_line("Hello, world!")

When we run the compiler again:

$ wingc hello.w
$ cat hello.txt
cat: hello.txt: No such file or directory
$ cat hello2.txt
Hello, world!

OK, this is getting interesting. We've changed the name of the file inside our wing program, re-run the compiler, and hello.txt doesn't exist anymore. This might not be what you would have expected because in most programming languages, if you use something like write_file(), it would have simply created another file with the new name.

Let's look at an even cooler example example2.w:

for i in 0..7 {
  let file = fs.TextFile("output/${i}/file-${i}.txt");
  file.add_line("hello, ${i}!");
}

And:

$ wingc example2.w
$ find output
output/0/file-0.txt
output/1/file-1.txt
output/2/file-2.txt
output/3/file-3.txt
output/4/file-4.txt
output/5/file-5.txt
output/6/file-6.txt

Now, let's say we change the for loop to loop over 0..2:

$ wingc example2.w
$ find output
output/0/file-0.txt
output/1/file-1.txt
output/2/file-2.txt

Magic! Our compiler actually deleted files based on the new definition in our program.

Reqtag: w:prune

One way to think about it is that wing programs define desired state through files, and the compiler manages these files across executions. This means that developers are able to evolve the desired state by simply updating their code.

To do that, the wing compiler maintains a state file (e.g. hello.w.state) which tells it which files were synthesized by the code. This file tells the compiler which files should be deleted across executions.

NOTE: The state file is not mandatory, and its location may differ based on the type of application. For example, if the output of the compiler all goes under some dist or target directory, then the state file is either redundant or can be placed inside that directory.

To ensure that users don't tamper with the files generated by wing, all generated files are always created as read-only, and their hash is stored in the state file. If a file is changed outside of wing, the compiler won't override it.

Reqtag: w:readonly

Resources​

Composition is the key to software abstraction. Breaking a problem into smaller pieces allows developers to focus on each piece independently and build complex systems by composing together and reusing pieces.

wing introduces the concept of resources as an object-oriented composition mechanism for desired-state which is based on the Construct Programming Model, which has been developed as part of the AWS CDK project. The CPM has been established as a powerful mechanism for modeling cloud resources through code and have been used to implement other desired-state frameworks such as CDK for Kubernetes, CDK for Terraform and Projen. The Construct Hub is central repository for sharing constructs for all CDKs, and we indend for wing resources to be part of this ecosystem.

Resources share the capabilities of classic object-oriented classes (such as initializers, methods, properties, inheritance, etc) but they have a very unique attribute that makes them suitable for defining desired-state through software - they have a deterministic address across compilations. In traditional object-oriented languages, instances of classes also have addresses, but these addresses are ephemral and only apply to a single execution of the program. Every time the program is executed, each object gets allocated in memory and gets a new address.

Being able to map a resource to the same resource across compilations is at the core of the construct programming model. wing leverages the constructs library which is the base library used by all CDKs and enables interoperability across the ecosystem.

Declaring and defining resources​

In wing, resources are defined (instantiated) like this:

let my_resource = MyResource()

Reqtag: w:resource-definition

As you may know, the first two initializer arguments for resources in programming languages like TypeScript, Java or Python, are scope and id. These two values are the key to allowing resources to be composed together and maintain a deterministic address across executions.

However, contrary to how resources are defined traditional languages, wing allows you in certain cases to omit the scope and id. This reduces cognitive overload and potential mistakes and makes wing cleaner to read and write.

resource NotifyingBucket {
  init() {
    let bucket = Bucket()
    let topic = Topic()
  }
}

As you can see above, when defining the Bucket and Topic inside the initializer of NotifyingBucket, we didn't need to specify their scope and id.

If not otherwise specified, wing will always use this as the scope and the type name as the id.

Reqtag: w:resource-default-scope

Reqtag: w:resource-default-id

For reference, this is the equivalent TypeScript version:

const bucket = new s3.Bucket(this, 'Bucket');
const topic = new sns.Topic(this, 'Topic');

This works in the majority of the cases, but can also be customized if needed.

To explicitly specify the resource identifier, use the be "ID" syntax:

let bucket = Bucket() be "MyBucket";

Reqtag: w:resource-custom-id

This is needed, for example, if there are multiple resources of the same type within the same scope:

let topics = mut_list<Topic>();
for i in 0..10 {
  topics.add(Topic() be "Topic-${i}")
}

Bear in mind that custom identifiers are still a scope-unique and not the global address of the resource. Controlling the global address of cloud resources is an engine-dependent API. For example, in the AWS CDK it is possible to override the CloudFormation logical name of a resource using the CfnResource.overrideLogicalId() method.

To explicitly specify the resource scope, use the in SCOPE syntax:

Topic() in alternative_scope

Reqtag: w:resource-custom-scope

This can be used, for example, to form resource trees on the fly. For example, in unit tests:

let app = aws.App();
let stack = aws.Stack() in app;
let my_bucket = s3.Bucket() be "MyBucket" in stack;

let template = app.synth().template;
expect(template).to(...);

Resource Tree Reflection​

One of the most powerful asepcts of the resource tree is that it offers a rich programming model for reflecting on the tree. The reflection API is available under the constructs.Node of each resource, and can be accessed via node_of(c):

Reqtag: w:resource-node

let node = node_of(my_bucket);
assert(node.scope == stack);
assert(node.id == "MyBucket");
assert(node.addr == "c876fd36dd614e466ada94591af0f00e3600fe3648");

The addr property of a [constructs.Node] is the program-unique deterministic address of this resource and used by synthesizers to produce logical identifiers (such as CloudFormation Logical IDs) which retain across executions.

Declaring resource types​

Similarly, when resource types are declared, wing the scope and id positional arguments are not explicitly passed to the type initializer, as well as the base constructs.Construct class.

In wing:

resource Foo {
  init() {

  }
}

The equivalent in TypeScript:

class Foo extends Construct {
  constructor(scope: Construct, id: string) {
    super(scope, id);

  }
}

Reqtag: w:resource-declaration

The root resource​

Every wing program has an implicit root resource which is the resource used as the scope for resources defined in the main program.

This is the equivalent of the CDK App construct.

Reqtag: w:resource-implicit-root

The compiler manipulates the root resource based on its compilation target.

For example, consider a simple wing program test.w, which defines a single bucket:

bring cloud;

cloud.Bucket();

And after we compile the code with the cloudformation target:

$ wingc test.w --target aws-cloudformation

We will get the file cdk.out/Default.template.json:

{
  "Resources": {
    "Bucket83908E77": {
      "Type": "AWS::S3::Bucket",
      "UpdateReplacePolicy": "Retain",
      "DeletionPolicy": "Retain"
    }
  }
}

Under the hood, the target basically tells the compiler how to configure the root resource so that cloud resources are resolved to AWS CloudFormation resources and the synthesis output is an AWS CloudFormation template.

Alternatively, if one would use the tf-gcp target, the output will be a Terraform configuration for deploying the bucket on Google Cloud Platform.

Reqtag: w:synthesis-target

The compiler is not opinionated about the concept of cloud providers (e.g. AWS, GCP, Azure) or provisioning engines (e.g. CloudFormation, Terraform, Pulumi), and users can supply user-defined implementation for the root resource which implement custom use cases such as multi-cloud/multi-engine deployments.

Initially, targets are implemented via a JavaScript module can be referenced like so:

$ wingc test.w --target npm://@acme/wing-targets@^1.2.3/Default

Reqtag: w:synthesis-target-custom

Asynchronous Finalizers​

There are use cases where a resource needs to execute an asynchronous operation just before the output is synthesized. An example use case is running an out of process bundler or some other 3rd party code that only has an async API:

Reqtag: w:resource-async-finalizer

resource MyResource {
  async fin() {
    // run some async code here
  }
}

It is important to note that the finalization order cannot be guarenteed, and finalizers should avoid mutating the tree because of that.

This is implemented by awaiting the underlying synth() method inside an async context in the preflight code generated by the wing compiler.

If wing will support explicit mutability, the compiler will be able to ensure that mutations don't happen during finalization.

Reqtag: w:resource-async-finalizer-immutability

Inflight Functions ("inflights")​

wing is a language for defining desired state through software. But desired-state is not just about configuration of e.g. cloud resources, it is also about defining the computation logic which executes inside these resources after the system is deployed (or even during deployment).

In wing, we differentiate between code that executes during compilation and code that executes after the system has been deployed by referring to them as "preflight" and "inflight" code respectively.

The ability to express runtime logic as an integral part of the desired-state definition, and naturally interact between the two execution domains is a unique and fundamental capability of the wing language.

The default (and implicit) execution context in wing is preflight. This is because in cloud applications, the entrypoint is actually the definition of the app's architecture (and not the code that runs within a specific machine within this system).

To support this, wing has a language primitive called "inflight functions" (in short "inflights") which represents an isolated computation unit that can be packaged and executed later on various compute platforms. Inflights are able to naturally interact with resources which are defined outside of the inflight, and invoke runtime operations on them.

Reqtag: w:inflight

Let's look at an example upload.w:

let bucket = cloud.Bucket();
let topic = cloud.Topic();
let message_count = 4;
let fn = cloud.Function(inflight (e: cloud.BucketUploadEvent) => {
  for i in 0..message_count {
    topic.publish("new file uploaded to our bucket: ${e.name} - ${i}");
  }
});
bucket.on_upload(fn);

It should be quite intuitive to understand what the above example is doing. It defines a bucket and when objects are uploaded to the bucket, it will publish a message on a topic with the object key.

As you can see, the upload_handler is declared via the inflight fn keyword and then passed into the cloud.Function as the first initializer argument (think of cloud.Function as an abstract version of AWS Lambda functions).

Inside upload_handler, you can see that we call topic.publish() which is part of the runtime API of the topic and basically publishes a message to the pub/sub topic with the name of the uploaded file.

When we compile this wing program, we will get something like this:

$ wingc upload.w --target cloudformation
$ find cdk.out
cdk.out/Default.template.json
cdk.out/asset.250ffecc3ad9d703e3df52ff035a1ec6ace6d2afaff290c6937b879f0897fc16.bundle/index.js

The output here includes both a CloudFormation template (with the infrastructure resource definitions) and an index.js file with the AWS Lambda handler code and all needed dependencies.

If you dive into the CloudFormation template and the runtime code, you'll see that the compiler took care of quite a lot of undifferentiated details in order to implement this application for AWS:

  1. Pass the ARN of the topic into the AWS Lambda function through an environment variable.
  2. Add sns:PublishMessage permissions to the AWS Lambda execution policy, with the least-privilege permissions for this specific topic.
  3. Include the client library of cloud.Topic with implementation for AWS in the AWS Lambda bundle.

Resources cannot be defined within inflight functions, because there is no synthesizer and no provisioning engine to deploy those resources.

Reqtag: w:inflight-no-resource-definitions

Capturing resources​

An inflight can naturally interact with the *Runtime API of resources that are defined outside of the inflight's code block (e.g. topic in the above example).

When a resource is captured by an inflight, there is an API that allows reflecting on the capture, and respond accordingly. For example, the code that implements cloud.Function(proc) will use this in order to identify which resources are captured by the inflight, and which methods are being called on its runtime API, so it can wire the desired information and add the appropriate permissions.

Reqtag: w:inflight-capture-resources

Capturing immutable data​

Immutable primitive values can also be referenced and captured by inflights. In such cases, the data will be copied and included into the bundled output as static values.

Primitive values which include tokens can be captured, and will be resolved during deployment by assigning them to environment variables (or other means of dynamic runtime values).

Reqtag: w:inflight-capture-data

Bear in mind that when capturing collections (e.g. lists or maps), they can reference resources as well, in which case we need to capture those resources.

Consider the following example:

let bucket_per_region = {
  US: Bucket(region: 'us-east-1'),
  EU: Bucket(region: 'eu-west-2'),
};

let handler = inflight (event) => {
  let bucket = bucket_per_region.get(event.region);
  await bucket.upload('boom', 'bam');
};

The handler above captures the two buckets in the map. However, since the actual interaction with the bucket (upload()) happens indirectly (only after the bucket is determined from the map), it will be difficult for the compiler to determine that the bucket.upload() call happens on a captured resource.

This might be possible in the future, but requires deep static analysis of the code (the compiler will have to understand that map.get() returns the contents of the value of the map).

If the compiler fails to determine the nature of the capture, it needs to emit an error:

let handler = inflight event => {
  let bucket = bucket_per_region.get(event.region);
                     ^--------- ERROR: map elements must be captured explicitly
  await bucket.upload('boom', 'bam');
};

And here's proposed syntax for these explicit captures:

let handler = inflight (event) => {
  let bucket = bucket_per_region.get(event.region);
  await bucket.upload('boom', 'bam');
} captures [
  { obj: bucket_per_region.US, methods: ["upload"] },
  { obj: bucket_per_region.NA, methods: ["upload"] },
];

Reqtag: w:inflight-explicit-capture

Indirect resource captures​

Consider this example:

resource DenyList {
  _bucket: cloud.Bucket;

  init() {
    this._bucket = cloud.Bucket();
  }

  inflight _map: any;

  inflight init() {
    this._map = this._bucket.download_json("deny-list.json");
  }

  inflight is_blocked(name: str, version: str): bool {
    return this._map[name] ?? this._map["${name}/v${version}"];
  }
}

let deny_list = DenyList();
let handler = inflight (event: any) => {
  if deny_list.is_blocked(event.name, event.version) {
    print("${event.name}@${event.version} is blocked");
  }
};

In the above example, handler captures deny_list which is a user-defined resource. Under the hood, this resource uses a bucket. When capturing deny_list, the compiler needs to implicitly capture the bucket behind it.

Reqtag: w:inflight-capture-indirect

Capturing mutable objects is not allowed​

The capturing rules above imply that normal objects cannot be referenced from within inflight closures, since objects in wing can technically be mutable via method calls.

This does not apply to resources because they become immutable after the system is deployed and when they are captured, we capture them through their runtime client.

TODO: we can consider having explicit immutability for objects and then we can allow marshalling immutable objects as well.

This won't compile:

class MyClass {
  mutate_me() {

  }
}

my_class := MyClass()

inflight fn bar() {
  my_class.mutate_me()
  // ^-- ERROR: trying to capture mutable object
}

Reqtag: w:inflight-capture-forbid-mutable

Static variables within inflight code​

It is not uncommon for inflight functions to need to hold state across executions. In some compute platforms such as AWS Lambda, such state can be used as a short-term cache.

Inflight functions support this via the static keyword (inspired from C):

inflight fn my_handler() {
  static big_blob := download_big_blob()

  // use big_blob
}

In the above example, the big_blob object will be defined as a global variable of the AWS Lambda function and will be preserved across executions within the same AWS Lambda server.

Reqtag: w:inflight-static

Dependency Injection​

One of the main goals of wing is to allow developers to write portable cloud applications. To enable this, wing supports defining resources that are abstract, and only during compilation, resolve their concrete implementation.

abstract resource Bucket {
  abstract make_public(): void

  make_public_and_print() {
    this.make_public()
    print("boom")
  }
}

my_bucket := Bucket()
my_bucket.make_public_and_print()

Now, if we compile this program:

$ wingc prog.w
prog.w:5:ERROR: unable to resolve abstract resource `Bucket`

The wing compiler basically tells us that it doesn't know how to resolve the abstract Bucket resource we used in our program. We need to "inject" an implementation for it when we compile.

The wing compiler supports resolving abstract definitions in multiple ways: via the compiler command line, a declarative resolution file or library or via additional code (TODO).

Sketch:

$ wingc prog.w --resolve "Bucket=aws.s3.Bucket"

Reqtag: w:dependency-injection

Runtime Client APIs​

As mentioned above, when an inflight function interacts with resources, it is only allowed to use their runtime API. One may think of this as the "client" of the resource which is how it is modeled in many systems.

Let's look at an example:

bring 'aws-sdk' as awssdk;
bring 'aws-cdk-lib' as awscdk;

resource Users {
  _table: awscdk.dynamodb.Table;

  // preflight initializer (constructor)
  init() {
    this._table := awscdk.dynamodb.Table(partition_key: "id");
  }

  inflight _client: awssdk.DynamoDB;

  // inflight initializer
  inflight init() {
    this._client := aws.DynamoDB();
  }

  // inflight "client"
  inflight add_user(id: string, name: string, last: string) {
    await this._client.put_item(
      TableName: this.table.table_name,
      Item: {
        ID: { S: id },
        UserName: { S: name },
        LastName: { S: last },
      },
    );
  }
}

let users = Users();
let new_users = cloud.Queue();
new_users.add_consumer(cloud.Function(inflight (e) => {
  users.add_user(e.user_id, e.name, e.last);
}));

Reqtag: w:inflight-clients

Observability​

Observability is a fundamental aspect of any application that runs on the cloud. wing (and its standard library) have out of the box support for various observability features:

  1. Metrics - metrics can be defined and reported with syntax (item_count++).
  2. Alarms - users are able to define alarms on any metric. The syntax of alarm definition is strongly-typed and compile-time checked.
  3. Logs - logs can be emitted from any inflight at runtime and can be tailed and viewed as a single flow across the entire system.
  4. Tracing - a trace identifier is implicitly passed to all network calls and observability tools can leverage it to provide a live trace of the system across distributed components.

wing embraces Open Telemetry.

Reqtag: w:observability-metrics

Reqtag: w:observability-alarms

Reqtag: w:observability-logs

Reqtag: w:observability-tracing

Opaque Primitives (Tokens)​

Tokens are opaque primitive types which include values that can only be resolved during deployment. The wing type system has built-in support for tokens in order to protect users from accidentally tampering with those values.

TODO: constraints are only during prefligt

Maybe we could implement this by having String derive from OpaqueString and then the latter will not have .length or splitting or reading the contents.^

bucket := Bucket()

print(bucket.name.length)
/// ----------------^
//                  can't take the length of an opaque value

inflight fn handler() {
  print(bucket.name.length)
  print(bucket.name)
}

Reqtag: w:tokens

Interoperability​

  • wing libraries are effectively JSII libraries

  • JSII libraries can be imported and used natively in wing code

  • CDK resources can be used naturally within wing resources and vice versa

  • It is possible to use any TypeScript library within wing. TypeScript type information will be used to offer strong-typing.

    • TODO: Use existing docker images/lambda bundles

Reqtag: w:interop-jsii

Type System​

  • Compatible with the JSII type system
  • Duration/Size literals (e.g 5s, 19GiB)
  • JSON literals

Reqtag: w:typesystem

Preflight Warnings and Errors​

wing preflight code is executed during compilation. This means that it can emit errors or warnings during that time, and they will be displayed as compiler diagnostics in compilation output and IDE tooling.

Let's look at this example:

struct MyResourceProps {
  max_len: number;
  name: string;
}

resource MyResource {
  init(props: MyResourceProps) {
    if props.name.length > props.max_len {
      throw("`name` is too long (${props.name.length} > ${props.max_len})")
    }

    if props.name.length == props.max_len {
      print("Be careful, your name is at the maximum length")
    }
  }
}

Then, say a consumer uses it like this:

let my = MyResource(name: "hello", max_len: 3)
//                  ^---- ERROR: `name` is too long (5 > 3)

This is actually a very powerful and common scenario, when there are some logical constraints that cannot be expressed via the type system but are an integral part of the contract (preconditions) for a certain type.

Open issues:

  • It should be possible to distinguish between a warning and an error.
  • From a control-flow perspective, it makes sense to use something like throw/raise to escape the flow when there is an error, but not when there is a warning.
  • See construct tree annotations in the AWS CDK as inspiration

Reqtag: w:preflight-errors

Requirement List​

Wishlist​

This is a list of features we will consider for wing as it evolves:-

  • Escpae Hatches - we will consider a built-in mechanism for escape hatching in wing.

  • REST, GraphQL and Microservices - wing allows developers to define GraphQL and REST endpoints using the type system and automatically generate OpenAPI or GraphQL specifications as well as multi-language client libraries. wing will reduce much of the boilerplate required to discover and interact across microservices by allowing two microservices to interact across API boundaries.

  • Workflows: wing allows preflight code to reflect on the code inside inflight blocks in order to convert it to definitions for distributed workflow engines such as AWS Step Functions or Apache Airflow. The functionless project is exploring this direction with TypeScript. This can also be used to generate things like CI/CD workflows such as GitHub Workflow.

  • Web3: Can wing be useful to build systems that include blockchain smart contracts and/or compile to Solidity.

  • Cloud data structures: wing will be able to offer first-class language primitives that implement data structures on the cloud. For example, a map can be implemented using a key-value store, a global variable can be implemented using a distributed counter, etc.

  • Frontend development: Websites are an integral part of cloud applications. As such the frontend logic is part of the app. We see a potential for wing to expand from the backend to also include the frontend logic and reduce the boilerplate and glue that exists today when crossing these domains.

References​