Eclipse Xtext

The Xtext adapter identifies elements of Xtext documents as traceable artifacts. The adapter can be configured based on the grammar rules of a domain-specific language (DSL) that has been defined with Xtext. The adapter propagates selected artifacts to and from Xtext files using the DSL-specific editor generated by Xtext.

Xtext allows for defining grammars for textual domain-specific languages (DSL) . In this section, the following definitions apply:

  • If D is a DSL defined by Xtext, a D document is a textual document that conforms to the grammar rules of language D.
  • An Xtext document is a textual document that conforms to any of the languages defined by Xtext. For example, if A, B, and C are DSLs defined by Xtext, A documents, B documents, and C documents are jointly called Xtext documents. Typically, when we are talking about Xtext documents, we are talking about general principles pertaining to all Xtext-defined languages, not about the peculiarities of a specific DSL.
  • An Xtext document element is a syntactical unit of an Xtext document, as defined by the grammar rules of the particular Xtext language. An Xtext document element might simply be called document element or element, as long as the context is unambiguous.

Data access

Configuration

Open the YT configuration with the YT configuration editor, and add a new data access as described in section "Data accesses". Select Xtext files as data access type.

Supported options:

  • resource – File filter pattern

Example:

resource A*.entity
resource *b.entity

These configuration rules specify that YT should load Xtext documents from files whose filenames – disregarding filename extensions – start with the letter A or end with the letter b, and whose filename extension is .entity.

Please note that the filename extension is mandatory. The Xtext adapter will only work if the Xtext grammar can be inferred from the filename extension. If you want to configure different several file filters for the same data access, all of them must have the same filename extension. Then again, if you want to trace artifacts of several different DSLs, you will have to define a separate data access for each Xtext grammar.

It is also possible to instruct YT to avoid parsing the Xtext files but read the artifacts from the Xtext index. This can be much faster but of course it can be read only what the language actually writes to the index. By default these are the entities that have a „name” of type ID, but this can be configured. What is written to the index can be controlled by an implementation of „org.eclipse.xtext.resource.IDefaultResourceDescriptionStrategy”. To configure this kind of reading place the keyword fromIndex right after the resource keyword.

Example:

resource fromIndex A*.entity

These configuration rules specify that YT should load Xtext documents from files whose filenames – disregarding filename extensions – start with the letter A but should avoid re-parsing of these files but access the index instead.

Artifact type

The Xtext artifact type identifies Xtext document elements as artifacts. These elements are identified based on which Xtext grammar rules they correspond to. Every document element corresponds to at least one rule that defines the context in which the element may occur.

Assume, for example, that your DSL D contains the following rules R, R1 and R2:

R: R1 | R2;

R1: 'X' x=STRING;

R2: 'Y' y=STRING;

If YT is configured to identify artifacts matching the rule R1, it will identify a document element X "text" as an artifact. If it is configured for R2, it will instead identify Y "text" as artifact. YT will recognize both as artifacts if it is configured for R, because it references R1 and R2 in unassigned rule calls (please refer to the Xtext documentation for this special Xtext construct).

For each rule of the Xtext grammar you want to derive document elements as artifacts from, you can define how the artifact name and the values of custom attributes should be computed. In both cases, you can specify expressions that refer to assignments, e.g., x=STRING in case of rule R in the sample grammar above. These assignments correspond to attributes or refer to document elements. For example, given the document element artifact X "text", matching the rule R1: 'X' x=STRING;, the value of attribute x is "text").

Configuration

Open the YT configuration with the YT configuration editor, and add a new artifact type as described in section "Artifact types". Select your previously-configured Xtext data access in the Data access drop-down list.

We will introduce the configuration language, including its syntax, by using an extensive example. You can find the complete list of keywords following the example.

A sample Xtext grammar

Let’s consider the following two rules of the (actually much larger) Xtext grammar for the YT query language:

Query:
	('query' name = STRING)?
	('description' description = STRING)?
	source=QSource
	collect=QCollect?
	where=QWhere?
	groupBy=QGroupBy?
	orderBy=QOrderBy?
;

QCollect:
	{QCollect}
	'.' 'collect' '(' features+=QFeatureSelectionWithAggregation (',' features+=QFeatureSelectionWithAggregation)* ')'
;

In this sample grammar, the rule Query represents a query. It references – among other things – the rule QCollect, which represents the optional collect statement. For this example, let’s consider the case that we want to examine query files and identify individual queries within them as artifacts.

We are using the following query file, comprising two queries, as a sample document of this grammar:

query "name" description "description of the query"
 source(someQueryFunction(parameter1, parameter2))
 .collect(attribute1 as A1, @sum(attribute2) as A2)
 .groupBy(attribute1)
 .orderBy(A2)

query "Requirements covered by test cases"
    source(tracesFromTo('Requirement', 'TestCase'))
    .collect(Start.DataSource.Identifier as FileName, Start.Name as ID)

The first steps are to

  • define a data access definition for the query files,
  • create an artifact type, and
  • assign the data access definition to it.

Here we want to search for queries in all files with the filename extension .query. This leads us to the following simple data access configuration:

*.query
Identifying artifacts by grammar rule

Next, we need to create a mapping and configure it. The following mapping configuration for an Xtext artifact type derives artifacts for the rule Query. In case of the query file above, two artifacts will be derived, one for each query. The configuration will be explained in more detail below.

rule Query {
    name "Query "+fqnOf(this)
    map {
        description to valueOf(description)
        collectedFeatures to valueOf(collect.features) joined with separator " / "
        startLine to startLineOf(this)
        endLine to endLineOf(this)
    }
}

The rule statement can refer to any rule defined by the grammar. In the example, it references the Query rule of the query grammar, however, it could for instance also reference the QCollect or the QSource rule. Press [Ctrl]+[Space] after rule to have the content assist propose all available grammar rules.

Let us assume that we require all artifacts to be named in a specific way. Furthermore, we need to extract some additional properties from each query and map them to the custom attributes of the corresponding artifact. We can specify the artifact name and the attribute mappings in the block within braces, following rule Query. Both, the mappings and the name definition, are optional. For this example, however, we need both.

While this example includes a single rule statement only, in general an Xtext artifact type configuration may contain several of them. This way, artifacts for different rules can be identified within the same configuration. Each of them has its own block with its own mappings that are specific to the corresponding rule. Note that there is only one set of custom attributes for each artifact type. Hence, all of an attribute type configuration’s rule blocks are sharing these attributes. Custom attributes, for which there is no mapping in a given block, will remain empty for all artifacts identified by its rule.

For this example, we want each artifact’s name to start with the character string „Query ”, followed by the fully-qualified name (FQN) of the corresponding query. We define this by using a name statement that appends the result of fqnOf(this) to the above-mentioned string. Here, this references the current query, and the function fqnOf(this) yields its FQN. The content assist proposals for the parameters of fqnOf also include all attributes of a query. These are defined by the assignments within the Query grammar rule, e.g., name, description, or source. For the sample query file above, the names of the derived artifacts are „Query name” and „Query Requirements covered by test cases”, because in this case the FQN is equivalent to the value of the attribute name.

Mapping document properties to artifact attributes

Now let’s consider mapping certain properties of each query to custom attributes of the artifact derived from it. We can do this similarly to many other artifact types: by using a map block. The following paragraph describes the individual properties and the mappings that we need for them.

First, we want to extract the description of the query, as defined by the description assignment in the grammar rule, and map it to a custom attribute of the same name. We do this in the first line of the map block by assigning valueOf(description) to the description attribute. Since description is not a document element that has nested children, the valueOf operator returns a string representation of the query’s description attribute value. In this case, however, the attribute type is STRING, which means that the attribute value is already a character string. For the sample query file, the first description is empty and the second is „description of the query”.

Next, we want to determine the list of features contained in the query’s collect statement and assign a string representation of this list to the custom attribute collectedFeatures. As you can see in the corresponding second line of the map block, there are several differences between this case and the above-mentioned mapping for description. We will discuss them in detail below.

Instead of directly reading an attribute of the current query, we need to access the features attribute of the query’s collect statement. It is defined by assignments to features in the QCollect rule. We can access this attribute using the expression collect.features. This expression first references the query attribute collect, which is an instance of QCollect, and then traverses to the features attribute, which contains a list of features. Then, the valueOf operator is applied to the features attribute in order to provide a character string.

As we just pointed out and contrary to description above, the features attribute is a list. Hence, valueOf determines a string for each element of this list and concatenates these strings. We want the list elements to be separated by the string " / " and achieve this by using the expression joined with separator " / ".

Each element of the list features is an instance of QFeatureSelectionWithAggregation, which is also defined in the query grammar, but not shown in the excerpt above. In the sample query document there are document elements matching the QFeatureSelectionWithAggregation rule. Here valueOf will not create a string based on the attribute value (i.e., the document element), but will instead yield the text representation of the document element within the query document. These texts will be concatenated as explained above.

In the sample query document, this works as follows. The collect statements we are going to process are:

  • collect(attribute1 as A1, @sum(attribute2) as A2)
  • collect(Start.DataSource.Identifier as FileName, Start.Name as ID)

The relevant expression in the artifact type configuration is:

  • valueOf(collect.features) joined with separator " / "

Applying this configuration, the collectedFeatures attributes of the query artifacts in the sample document have to following values:

  • attribute1 as A1 / @sum(attribute2) as A2
  • Start.DataSource.Identifier as FileName / Start.Name as ID

Finally, we also want to know the numbers of the first and the last lines of the queries, respectively, within the query document. The last two statements of the map block assign these line numbers to corresponding custom attributes. The keywords startLineOf and endLineOf find the first and the last line, respectively, of the document element given as their respective parameter. We again use the keyword this to refer to the query object itself. For the sample query document, the line numbers assigned are 1 and 5 for the first query and 7 and 9 for the second one.

A similar configuration to read queries from the index would look like this:

eClass Query {
    name "Query "+fqnOf(this)
    map {
        validation to userData("isValidationQuery")
    }
}

The above utilizes the fact that the query language writes a boolean information to the userData isValidationQuery in case it is a validation query. For your own language you should write the information you want to see for the artifacts into the index and then use userData to read them into the artifacts. Instead of rules you reference the types ( EClass) that is written to the index.

Configuration keywords

The Xtext artifact type configuration supports the following keywords:

  • endLineOf( elem) – Refers to the last line of the textual representation of an element within an Xtext document. The document element is addressed by the parameter elem as explained below. This function does not work in case your artifacts are read from the index because the information is usually not available in the index. If you need this information you could adapt your language to write this information to the index and use userData te read it from there.
  • fqnOf( elem) – Refers to the fully-qualified name (FQN) of a document element. The document element is addressed by the parameter elem as explained below.
  • joined with separator sep – Follows one of the ...Of expressions and allows to join (concatenate) the elements of a list into a single string, separated by the specified separator sep. Without this keyword, the separator ", " will be used. Document elements are part of a list if they correspond to assignments of the form variable += expression in the Xtext grammar. Here variable is the list and its elements are determined by expression.
  • map – Maps artifact properties to custom attributes. For each attribute it contains an expression of the form attribute to expression, where attribute is the attribute name and expression defines how the assigned value is computed.
  • name expr – Specifies the name of the artifact as the value the expression expr evaluates to. By default, the FQN is used, if defined. Otherwise, the fragment of the element’s URI is used. The fragment identifies the element locally within the document. In case of the queries example above, the fragment of the second feature in the collect statement of the first query would be „//@queries.0/@collect/@features.1”.
  • rule rulename – The Xtext rule, identified by rulename, for which document elements should be identified as artifacts (see above for examples). You can configure YT to derive elements for rules of the Xtext grammar that you have configured using the Xtext data access. Additionally, you can also refer to rules inherited from other Xtext grammars.
  • startLineOf( elem) – Refers to the first line of the textual representation of a document element. The document element is addressed by the parameter elem as explained below. This function does not work in case your artifacts are read from the index because the information is usually not available in the index. If you need this information you could adapt your language to write this information to the index and use userData te read it from there.
  • this – Refers to the current document element, which corresponds to the Xtext rule referenced by the rule block in which this is used. YT evaluates the rule block for every such element to create an artifact for it. In a given evaluation, this refers to the element that is considered in that evaluation. For example, in rule R { name valueOf(this) }, the keyword this can be any document element corresponding to the Xtext rule R.
  • valueOf( elem) – Refers to the textual representation of a document element. The document element is addressed by the parameter elem as explained below. If the attribute is a plain type, e.g., a string or number, or if it is defined externally to the document, e.g., in a document of another language, the object’s string value will be used.
  • userData( elem) – In case your elements are read from the index this function can be used to read custom information that is written by the language to the index. Usually a simple string is provided as an argument to specify which user data field should be read.
The elem parameter

The keywords valueOf, fqnOf, startLineOf, and endLineOf each have one parameter that denotes a document element. It can be this to reference the document element that is matching the current grammar rule. Alternatively, it can be an object navigational expression of the form object_1. object_2.. object_n. object , where object_1 is an attribute of this, object_2 is an attribute of object_1 and so on.

If the last object is a list, the respective keyword will be applied to each element individually, and the results will be concatenated by ", " or as indicated by joined with separator. For example, a list ("one", "two", "three") would be concatenated to "one.two.three" when using "." as a separator.

In case of valueOf, the final object can be any object, while the other …Of keywords only work for document elements (or lists of document elements) and will otherwise evaluate to an empty string. The keyword fqnOf only works for document elements that actually have an FQN. The other keywords only work for elements defined in the Xtext document.

Content assist supports you in the selection of Xtext rules. Its proposals are based on the Xtext grammar of the language that is configured for the respective data access. You can also use content assist to select attributes or references within valueOf. The proposals include attributes and references that are defined by any of the assignments defined for the given Xtext rule. For example, if the rule contains x=STRING and y=STRING, then the proposals contain x and y. If these assignments also correspond to further rules, you can recursively navigate through the assignments by chaining them with the dot operator ( .).

For the case that your elements are read from the index elem refers to the IEObjectDescription read from the index. This is a shallow representation of your object as written to the index. It includes the userData as written by the language but no structured information such as child elements.

Referencing elements from external languages and models

Please note that the content assist’s proposals won’t include attributes or references of other Xtext grammars, except for those that are inherited from Xtext grammars which are (directly or indirectly) included using the Xtext keyword „with”. Content assist for other EMF models – see Eclipse Modeling Framework – will only work if the configured Xtext grammar references elements of a specific type defined in the EMF model.

If there is no content assist for certain attributes or references you can reference them by manually typing their respective names without the help of the content assist. For example, if you know that an attribute object of type SuperType always contains an object of type SubType and that this type defines an attribute called name, then you can access this attribute by object."name". The quotation marks are needed here, because name is a keyword.

Version

An artifact’s version is used for suspicious links validation. Artifacts of this type do not provide a version.