In this example we will analyze the Dutch insurance market with two machine learning algorithms, t-SNE, a technique for dimensionality reduction developed by Laurens van der Maat, combined with KMeans, an algorithm to find clusters in the data.

We use publicly available register data of all Dutch insurance undertakings that we web scraped from the DNB website (public-register). This register contains the specific license and the lines of business (LoB’s) in which an insurance undertaking is allowed to operate.

Insurance undertakings in the European Union have different types of licenses, for example life or non-life and where the undertaking is based and where it is allowed to operate. They sell different kinds of insurance products, i.e. they operate in different LoB’s (for example motor vehicle insurance, general life insurance or health insurance). There are 6 LoB’s for life and 19 LoB’s for non-life.

There are groups of insurance undertakings that, looking at their lines of business, are similar. For example some undertakings are small and specialized and sell products only within a very limited number of lines of business. Other larger general insurance undertakings sell insurance products from all lines of business.

How can we find these clusters (groups) of insurance undertakings that are similar with respect to their set of lines of business? We will create a vector per insurance undertakings with their allowed lines of business. Then we will use the t-SNE algorithm to reduce this vector to a 2d vector such that we can plot it in a 2d plane. Undertakings with similar LoB sets are then plotted near to each other. Then we detect the clusters with the kMeans algorithm.

I won’t give all the Python code of the notebook because it is somewhat cumbersome to obtain the license data from the DNB register, but if you are interested, a part can be found here.

import pandas as pd
import numpy as np
import ast
import matplotlib.pyplot as pyplot
from sklearn.manifold import TSNE
from sklearn.cluster import KMeans

Step 1: reading the data

Earlier, we stored the license and lines of business data in a csv file.

filename = 'data/licenses_insurers.csv'
df = pd.read_csv(filename, delimiter = ',', encoding = 'utf-8')

Step 2: data preparation

First we need some data preparation. The .csv contains string data that has to be interpreted. First we extract the data on the licenses and the lines of business. If we look at the available licenses we get:

List of available licenses:
2:27 lid 1 SII-schadeverzekeraar
2:27(1) SII insurer
2:45(1) Carrying on the business of a life insurer (non-EEA)
2:45(1) Carrying on the business of a non-life insurer (non-EEA)
2:45(1) non-EEA-based life insurer providing services to NL
2:45(1) non-EEA-based non-life insurer providing services to NL
2:48(1) life insurer with low magnitude of risk
2:48(1) non-life insurer with low magnitude of risk
2:48(1) pre-paid funeral services insurer with low magnitude of risk
Section 1:104(3) Business being wound up

Apparently not all items have been translated to English.

Insurers with low magnitude of risk are too small for Solvency II regulation and for these insurers there is a (simplified) regime in place. Some entities are active from outside the European Economic Area (EEA). The codes refer to the articles in the Dutch Financial Supervision Act.

Now we can get the list of lines of business:

List of available lines of business:
L01. Life insurance - general
L02. Life insurance related to marriage or birth
L03. Life insurance linked to common funds
L05. Holdings in savings pools
L06. Capitalisation activities
L07. Collective pension funds management
S01. Accident insurance
S02. Health insurance
S03. Motor vehicle insurance
S04. Railway rolling stock insurance
S05. Aircraft hull insurance
S06. Marine hull insurance
S07. Goods-in-transit insurance
S08. Fire and natural forces insurance
S09. Other property damage insurance
S10a. Motor vehicle liability insurance
S10b. Road transport liability insurance
S11. Aircraft liability insurance
S12. Marine liability insurance (sea, lake & river and canal vessels)
S13. General liability insurance
S14. Credit insurance
S15. Suretyship
S16. Pecuniary loss insurance
S17. Legal assistance insurance
S18. Assistance

This list of lines of business is applied across the European Union and was already in place before the Solvency II regulation.

We can obtain a data frame df_entities_lobs with all insurance undertakings with their allowed lines of business (25 possible LoB’s). That will be the input for the t-SNE algorithm.

Step 3: data analysis

We will use the t-SNE algorithm from the package sklearn.manifold. The input for the algorithm is the data frame with dummy encoded lines of business per entity prepared previously.

X = df_entities_lobs
Y = TSNE(n_components = 2, 
         perplexity = 18, 
         verbose = 1, 
         random_state = 1).fit_transform(X)

Next we use the k-means algorithm to determine the clusters in the Dutch insurance market. It appears that there are about eight clusters that are identifiable.

kmeans = KMeans(n_clusters = 8, random_state = 0, n_init  = 10).fit(Y)

Step 4: data visualization

The last step is to visualize the results of the t-SNE algorithm. For this we first produce the labels of the clusters with the average number of lines of business and the two most dominant lines of business. With that information we can describe the basic properties of the cluster.

Cluster 1 and 5 are the two clusters with life insurance undertakings. Cluster 5 consists of general life insurance undertakings with a broad line of products in difference life insurance lines of business. Cluster 1 consists of specialized life insurance undertakings with only one line of business. These undertakings are often relatively small.

The other clusters are consists of non-life insurance undertakings. Cluster 2 is an easily identifiable cluster with health insurance undertakings, with exacly two lines of business (accident and health insurance). Some health insurance undertakings also have some other lines of business; they form more general health insurance undertakings cluster 6 (near the uniform cluster 2).

Then we have cluster 3 with general non-life insurance undertakings operating a large number of lines of business (13 on average). Cluster 7 consists of medium non-life insurance undertakings with less lines of business on average. And cluster 4 and cluster 0 are specialized and often small non-life insurance undertaking with 1 or 2 lines of business on average (cluster 0: specialized property damage and fire insurance undertakings, and cluster 4: specialized general liability and legal assistance insurance undertakings).

We can find the corresponding undertakings for each cluster. For example the specialized small life insurance undertaking (cluster 1):

list(df_entities_lobs.index[kmeans.labels_ == 1])

['DELA Natura- en levensverzekeringen N.V.',
 'Isle of Man Assurance Limited',
 'Monuta Verzekeringen N.V.',
 'N.V. Noordhollandsche van 1816, Levensverzekeringsmaatschappij',
 'Nordben Life and Pension Insurance Co. Limited',
 "Onderling Fonds 'Sliedrecht' B.A.",
 'Tiels Onderling Fonds tot Uitkering bij Overlijden Gustaaf Adolf U.A.',
 'Yarden Uitvaartverzekeringen N.V.',
 'Zurich Life Insurance Company Limited']