Matching Contest

Background

Process model matching refers to the creation of an alignment between process models, i.e., the creation of correspondences between their activities. In recent years, many techniques building on process model matching have been proposed. Examples include techniques for the validation of a technical implementation of a business process against a business-centered specification model, delta-analysis of process implementations and a reference model, harmonization of process variants, process model search, or clone detection. Inspired by the field of schema matching and ontology alignment, this demand led to the development of different techniques for process model matching. Yet, these techniques are heuristics and, thus, their results are inherently uncertain and need to be evaluated on a common basis. Currently, however, the BPM community lacks established datasets and frameworks for evaluation. This contest aims at addressing the need for effective evaluation. Towards this end, the contest defines process model matching problems, publishes datasets for these problems, and seeks implementations that solve them.

Matching Problems

The contest includes two sets of process model matching problems.

1. University Admission Processes. This set contains process models representing the admission processes of nine German universities. All models contain English text only. The models have been created by different modelers using varying terminology and capturing activities at different levels of granularity. All models are available as Petri-nets in the terminology and capturing activities at different levels of granularity. All models are available as Petri-nets in the PNML format and shall be matched pairwise. Further, for eight out of 36 model pairs, we also provide a gold standard alignement for initial evaluation.
2. Birth Registration Processes. This set comprises nine models of birth registration processes in Germany, Russia, South Africa, and the Netherlands. Four models were created by graduate students at the HU Berlin and five of the models stem from a process analysis in Dutch municipalities. Again, all models contain only English text, are available as Petri-nets in the PNML format, and shall be matched pairwise to obtain 36 alignments.

Participation

In order to take part in the matching contest, participants are asked to solve the given matching problems with their self-developed matching technique.

The Model Matching Technique (PDF)

The Triple-S matching approach adheres to the KISS principle by avoiding complexity and keeping it simple and stupid. It combines similarity scores of independent levels as basis for a well-founded decision about matching transition pairs of different process models. The following three levels and scores are considered:

• Syntactic level - SIM_syn(a, b)
• Semantic level - SIM_sem(a, b) and
• Structural level - SIM_struc(a, b).

These three scores are combined to the final score which represents the matching degree between two transitions a and b from different process models:

SIM_total(a, b) = ω_{1 *} SIM_syn(a, b) + ω₂ * SIM_sem(a, b) + ω₃ * SIM_struc(a, b).

The three parameters ω_1, ω₂, ω₃ define the weight of each similarity level. A
threshold value θ is used to determine whether transitions actually match, i.e. iff
SIM_total > θ, two transitions positively match.

For the contest, ω₁ = 0.45, ω₂ = 0.3,ω₃ = 0.25 and θ = 0.6 have been chosen to approximate optimal results based on the gold standard examples. With those values we achieve values of 0.49 and 0.35 for precision and recall for the given gold standard examples.

Download Page: Triple-S matching implementation

Further Information: PMC-MR' 13