Convolutional layers

The following convolutional/message-passing layers are available in Spektral.

Notation:

• $N$: number of nodes;
• $F$: size of the node attributes;
• $S$: size of the edge attributes;
• $\x_i$: node attributes of the i-th node;
• $\e_{i \rightarrow j}$: edge attributes of the edge from node i to node j;
• $\A$: adjacency matrix;
• $\X$: node attributes matrix;
• $\E$: edge attributes matrix;
• $\D$: degree matrix;
• $\W, \V$: trainable weights matrices;
• $\b$: trainable bias vector;
• $\mathcal{N}(i)$: one-hop neighbourhood of node $i$;

---

[source]

MessagePassing

spektral.layers.MessagePassing(aggregate='sum')

A general class for message passing networks from the paper

Neural Message Passing for Quantum Chemistry
Justin Gilmer et al.

Mode: single, disjoint.

This layer and all of its extensions expect a sparse adjacency matrix.

This layer computes: $$\x_i' = \gamma \left( \x_i, \square_{j \in \mathcal{N}(i)} \, \phi \left(\x_i, \x_j, \e_{j \rightarrow i} \right) \right),$$

where $\gamma$ is a differentiable update function, $\phi$ is a differentiable message function, $\square$ is a permutation-invariant function to aggregate the messages (like the sum or the average), and $\E_{ij}$ is the edge attribute of edge j-i.

By extending this class, it is possible to create any message-passing layer in single/disjoint mode.

API

propagate(x, a, e=None, **kwargs)

Propagates the messages and computes embeddings for each node in the graph.
Any kwargs will be forwarded as keyword arguments to message(), aggregate() and update().

message(x, **kwargs)

Computes messages, equivalent to $\phi$ in the definition.
Any extra keyword argument of this function will be populated by propagate() if a matching keyword is found.
The get_sources and get_targets built-in methods can be used to automatically retrieve the node attributes of nodes that are sending (sources) or receiving (targets) a message. If you need direct access to the edge indices, you can use the index_sources and index_targets attributes.

aggregate(messages, **kwargs)

Aggregates the messages, equivalent to $\square$ in the definition.
The behaviour of this function can also be controlled using the aggregate keyword in the constructor of the layer (supported aggregations: sum, mean, max, min, prod).
Any extra keyword argument of this function will be populated by propagate() if a matching keyword is found.

update(embeddings, **kwargs)

Updates the aggregated messages to obtain the final node embeddings, equivalent to $\gamma$ in the definition.
Any extra keyword argument of this function will be populated by propagate() if a matching keyword is found.

Arguments:

• aggregate: string or callable, an aggregation function. This flag can be used to control the behaviour of aggregate() wihtout re-implementing it. Supported aggregations: 'sum', 'mean', 'max', 'min', 'prod'. If callable, the function must have the signature foo(updates, indices, n_nodes) and return a rank 2 tensor with shape (n_nodes, ...).
• kwargs: additional keyword arguments specific to Keras' Layers, like regularizers, initializers, constraints, etc.

---

[source]

AGNNConv

spektral.layers.AGNNConv(trainable=True, aggregate='sum', activation=None)

An Attention-based Graph Neural Network (AGNN) from the paper

Attention-based Graph Neural Network for Semi-supervised Learning
Kiran K. Thekumparampil et al.

Mode: single, disjoint, mixed.

This layer expects a sparse adjacency matrix.

This layer computes: $$\X' = \P\X$$ where $$\P_{ij} = \frac{ \exp \left( \beta \cos \left( \x_i, \x_j \right) \right) }{ \sum\limits_{k \in \mathcal{N}(i) \cup \{ i \}} \exp \left( \beta \cos \left( \x_i, \x_k \right) \right) }$$ and $\beta$ is a trainable parameter.

Input

• Node features of shape (n_nodes, n_node_features);
• Binary adjacency matrix of shape (n_nodes, n_nodes).

Output

• Node features with the same shape of the input.

Arguments

• trainable: boolean, if True, then beta is a trainable parameter. Otherwise, beta is fixed to 1;
• activation: activation function;

---

[source]

APPNPConv

spektral.layers.APPNPConv(channels, alpha=0.2, propagations=1, mlp_hidden=None, mlp_activation='relu', dropout_rate=0.0, activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)

The APPNP operator from the paper

Predict then Propagate: Graph Neural Networks meet Personalized PageRank
Johannes Klicpera et al.

Mode: single, disjoint, mixed, batch.

This layer computes: $$\Z^{(0)} = \textrm{MLP}(\X); \\ \Z^{(K)} = (1 - \alpha) \hat \D^{-1/2} \hat \A \hat \D^{-1/2} \Z^{(K - 1)} + \alpha \Z^{(0)},$$ where $\alpha$ is the teleport probability, $\textrm{MLP}$ is a multi-layer perceptron, and $K$ is defined by the propagations argument.

Input

• Node features of shape ([batch], n_nodes, n_node_features);
• Modified Laplacian of shape ([batch], n_nodes, n_nodes); can be computed with spektral.utils.convolution.gcn_filter.

Output

• Node features with the same shape as the input, but with the last dimension changed to channels.

Arguments

• channels: number of output channels;
• alpha: teleport probability during propagation;
• propagations: number of propagation steps;
• mlp_hidden: list of integers, number of hidden units for each hidden layer in the MLP (if None, the MLP has only the output layer);
• mlp_activation: activation for the MLP layers;
• dropout_rate: dropout rate for Laplacian and MLP layers;
• activation: activation function;
• use_bias: bool, add a bias vector to the output;
• kernel_initializer: initializer for the weights;
• bias_initializer: initializer for the bias vector;
• kernel_regularizer: regularization applied to the weights;
• bias_regularizer: regularization applied to the bias vector;
• activity_regularizer: regularization applied to the output;
• kernel_constraint: constraint applied to the weights;
• bias_constraint: constraint applied to the bias vector.

---

[source]

ARMAConv

spektral.layers.ARMAConv(channels, order=1, iterations=1, share_weights=False, gcn_activation='relu', dropout_rate=0.0, activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)

An Auto-Regressive Moving Average convolutional layer (ARMA) from the paper

Graph Neural Networks with convolutional ARMA filters
Filippo Maria Bianchi et al.

Mode: single, disjoint, mixed, batch.

This layer computes: $$\X' = \frac{1}{K} \sum\limits_{k=1}^K \bar\X_k^{(T)},$$ where $K$ is the order of the ARMA$_K$ filter, and where: $$\bar \X_k^{(t + 1)} = \sigma \left(\tilde \A \bar \X^{(t)} \W^{(t)} + \X \V^{(t)} \right)$$ is a recursive approximation of an ARMA$_1$ filter, where $\bar \X^{(0)} = \X$ and $$\tilde \A = \D^{-1/2} \A \D^{-1/2}.$$

Input

• Node features of shape ([batch], n_nodes, n_node_features);
• Normalized and rescaled Laplacian of shape ([batch], n_nodes, n_nodes); can be computed with spektral.utils.convolution.normalized_laplacian and spektral.utils.convolution.rescale_laplacian.

Output

• Node features with the same shape as the input, but with the last dimension changed to channels.

Arguments

• channels: number of output channels;
• order: order of the full ARMA$_K$ filter, i.e., the number of parallel stacks in the layer;
• iterations: number of iterations to compute each ARMA$_1$ approximation;
• share_weights: share the weights in each ARMA$_1$ stack.
• gcn_activation: activation function to compute each ARMA$_1$ stack;
• dropout_rate: dropout rate for skip connection;
• activation: activation function;
• use_bias: bool, add a bias vector to the output;
• kernel_initializer: initializer for the weights;
• bias_initializer: initializer for the bias vector;
• kernel_regularizer: regularization applied to the weights;
• bias_regularizer: regularization applied to the bias vector;
• activity_regularizer: regularization applied to the output;
• kernel_constraint: constraint applied to the weights;
• bias_constraint: constraint applied to the bias vector.

---

[source]

CensNetConv

spektral.layers.CensNetConv(node_channels, edge_channels, activation=None, use_bias=True, kernel_initializer='glorot_uniform', node_initializer='glorot_uniform', edge_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, node_regularizer=None, edge_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, node_constraint=None, edge_constraint=None, bias_constraint=None)

A CensNet convolutional layer from the paper

Co-embedding of Nodes and Edges with Graph Neural Networks
Xiaodong Jiang et al.

This implements both the node and edge propagation rules as a single layer.

Mode: single, disjoint, batch.

Input

• Node features of shape ([batch], n_nodes, n_node_features);
• A tuple containing:
• Modified Laplacian of shape ([batch], n_nodes, n_nodes); can be computed with spektral.utils.convolution.gcn_filter.
• Modified line graph Laplacian of shape ([batch], n_edges, n_edges); can be computed with spektral.utils.convolution.line_graph and spektral.utils.convolution.gcn_filter.
• Incidence matrix of shape ([batch], n_nodes, n_edges); can be computed with spektral.utils.convolution.incidence_matrix.
• Edge features of shape ([batch], n_edges, n_edge_features);

Output

• Node features with the same shape as the input, but with the last dimension changed to node_channels.
• Edge features with the same shape as the input, but with the last dimension changed to edge_channels.

Arguments

• node_channels: number of output channels for the node features;
• edge_channels: number of output channels for the edge features;
• activation: activation function;
• use_bias: bool, add a bias vector to the output;
• kernel_initializer: initializer for the weights;
• node_initializer: initializer for the node feature weights (P_n);
• edge_initializer: initializer for the edge feature weights (P_e);
• bias_initializer: initializer for the bias vector;
• kernel_regularizer: regularization applied to the weights;
• edge_regularizer: regularization applied to the edge feature weights;
• bias_regularizer: regularization applied to the bias vector;
• activity_regularizer: regularization applied to the output;
• kernel_constraint: constraint applied to the weights;
• edge_constraint: constraint applied to the edge feature weights;
• bias_constraint: constraint applied to the bias vector.

---

[source]

ChebConv

spektral.layers.ChebConv(channels, K=1, activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)

A Chebyshev convolutional layer from the paper

Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering
Michaël Defferrard et al.

Mode: single, disjoint, mixed, batch.

This layer computes: $$\X' = \sum \limits_{k=0}^{K - 1} \T^{(k)} \W^{(k)} + \b^{(k)},$$ where $\T^{(0)}, ..., \T^{(K - 1)}$ are Chebyshev polynomials of $\tilde \L$ defined as $$\T^{(0)} = \X \\ \T^{(1)} = \tilde \L \X \\ \T^{(k \ge 2)} = 2 \cdot \tilde \L \T^{(k - 1)} - \T^{(k - 2)},$$ where $$\tilde \L = \frac{2}{\lambda_{max}} \cdot (\I - \D^{-1/2} \A \D^{-1/2}) - \I.$$

Input

• Node features of shape ([batch], n_nodes, n_node_features);
• A list of K Chebyshev polynomials of shape [([batch], n_nodes, n_nodes), ..., ([batch], n_nodes, n_nodes)]; can be computed with spektral.utils.convolution.chebyshev_filter.

Output

• Node features with the same shape of the input, but with the last dimension changed to channels.

Arguments

• channels: number of output channels;
• K: order of the Chebyshev polynomials;
• activation: activation function;
• use_bias: bool, add a bias vector to the output;
• kernel_initializer: initializer for the weights;
• bias_initializer: initializer for the bias vector;
• kernel_regularizer: regularization applied to the weights;
• bias_regularizer: regularization applied to the bias vector;
• activity_regularizer: regularization applied to the output;
• kernel_constraint: constraint applied to the weights;
• bias_constraint: constraint applied to the bias vector.

---

[source]

CrystalConv

spektral.layers.CrystalConv(aggregate='sum', activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)

A crystal graph convolutional layer from the paper

Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties
Tian Xie and Jeffrey C. Grossman

Mode: single, disjoint, mixed.

This layer expects a sparse adjacency matrix.

This layer computes: $$\x_i' = \x_i + \sum\limits_{j \in \mathcal{N}(i)} \sigma \left( \z_{ij} \W^{(f)} + \b^{(f)} \right) \odot \g \left( \z_{ij} \W^{(s)} + \b^{(s)} \right)$$ where $\z_{ij} = \x_i \| \x_j \| \e_{ji}$, $\sigma$ is a sigmoid activation, and $g$ is the activation function (defined by the activation argument).

Input

• Node features of shape (n_nodes, n_node_features);
• Binary adjacency matrix of shape (n_nodes, n_nodes).
• Edge features of shape (num_edges, n_edge_features).

Output

• Node features with the same shape of the input.

Arguments

• activation: activation function;
• use_bias: bool, add a bias vector to the output;
• kernel_initializer: initializer for the weights;
• bias_initializer: initializer for the bias vector;
• kernel_regularizer: regularization applied to the weights;
• bias_regularizer: regularization applied to the bias vector;
• activity_regularizer: regularization applied to the output;
• kernel_constraint: constraint applied to the weights;
• bias_constraint: constraint applied to the bias vector.

---

[source]

DiffusionConv

spektral.layers.DiffusionConv(channels, K=6, activation='tanh', kernel_initializer='glorot_uniform', kernel_regularizer=None, kernel_constraint=None)

A diffusion convolution operator from the paper

Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting
Yaguang Li et al.

Mode: single, disjoint, mixed, batch.

This layer expects a dense adjacency matrix.

Given a number of diffusion steps $K$ and a row-normalized adjacency matrix $\hat \A$, this layer calculates the $q$-th channel as: $$\mathbf{X}_{~:,~q}' = \sigma\left( \sum_{f=1}^{F} \left( \sum_{k=0}^{K-1} \theta_k {\hat \A}^k \right) \X_{~:,~f} \right)$$

Input

• Node features of shape ([batch], n_nodes, n_node_features);
• Normalized adjacency or attention coef. matrix $\hat \A$ of shape ([batch], n_nodes, n_nodes); Use DiffusionConvolution.preprocess to normalize.

Output

• Node features with the same shape as the input, but with the last dimension changed to channels.

Arguments

• channels: number of output channels;
• K: number of diffusion steps.
• activation: activation function $\sigma$; ($\tanh$ by default)
• kernel_initializer: initializer for the weights;
• kernel_regularizer: regularization applied to the weights;
• kernel_constraint: constraint applied to the weights;

---

[source]

ECCConv

spektral.layers.ECCConv(channels, kernel_network=None, root=True, activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)

An edge-conditioned convolutional layer (ECC) from the paper

Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs
Martin Simonovsky and Nikos Komodakis

Mode: single, disjoint, batch, mixed.

In single, disjoint, and mixed mode, this layer expects a sparse adjacency matrix. If a dense adjacency is given as input, it will be automatically cast to sparse, which might be expensive.

This layer computes: $$\x_i' = \x_{i} \W_{\textrm{root}} + \sum\limits_{j \in \mathcal{N}(i)} \x_{j} \textrm{MLP}(\e_{j \rightarrow i}) + \b$$ where $\textrm{MLP}$ is a multi-layer perceptron that outputs an edge-specific weight as a function of edge attributes.

Input

• Node features of shape ([batch], n_nodes, n_node_features);
• Binary adjacency matrices of shape ([batch], n_nodes, n_nodes);
• Edge features. In single mode, shape (num_edges, n_edge_features); in batch mode, shape (batch, n_nodes, n_nodes, n_edge_features).

Output

• node features with the same shape of the input, but the last dimension changed to channels.

Arguments

• channels: integer, number of output channels;
• kernel_network: a list of integers representing the hidden neurons of the kernel-generating network;
• 'root': if False, the layer will not consider the root node for computing the message passing (first term in equation above), but only the neighbours.
• activation: activation function;
• use_bias: bool, add a bias vector to the output;
• kernel_initializer: initializer for the weights;
• bias_initializer: initializer for the bias vector;
• kernel_regularizer: regularization applied to the weights;
• bias_regularizer: regularization applied to the bias vector;
• activity_regularizer: regularization applied to the output;
• kernel_constraint: constraint applied to the weights;
• bias_constraint: constraint applied to the bias vector.

---

[source]

EdgeConv

spektral.layers.EdgeConv(channels, mlp_hidden=None, mlp_activation='relu', aggregate='sum', activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)

An edge convolutional layer from the paper

Dynamic Graph CNN for Learning on Point Clouds
Yue Wang et al.

Mode: single, disjoint, mixed.

This layer expects a sparse adjacency matrix.

This layer computes for each node $i$: $$\x_i' = \sum\limits_{j \in \mathcal{N}(i)} \textrm{MLP}\big( \x_i \| \x_j - \x_i \big)$$ where $\textrm{MLP}$ is a multi-layer perceptron.

Input

• Node features of shape (n_nodes, n_node_features);
• Binary adjacency matrix of shape (n_nodes, n_nodes).

Output

• Node features with the same shape of the input, but the last dimension changed to channels.

Arguments

• channels: integer, number of output channels;
• mlp_hidden: list of integers, number of hidden units for each hidden layer in the MLP (if None, the MLP has only the output layer);
• mlp_activation: activation for the MLP layers;
• activation: activation function;
• use_bias: bool, add a bias vector to the output;
• kernel_initializer: initializer for the weights;
• bias_initializer: initializer for the bias vector;
• kernel_regularizer: regularization applied to the weights;
• bias_regularizer: regularization applied to the bias vector;
• activity_regularizer: regularization applied to the output;
• kernel_constraint: constraint applied to the weights;
• bias_constraint: constraint applied to the bias vector.

---

[source]

GATConv

spektral.layers.GATConv(channels, attn_heads=1, concat_heads=True, dropout_rate=0.5, return_attn_coef=False, add_self_loops=True, activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', attn_kernel_initializer='glorot_uniform', kernel_regularizer=None, bias_regularizer=None, attn_kernel_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None, attn_kernel_constraint=None)

A Graph Attention layer (GAT) from the paper

Graph Attention Networks
Petar Veličković et al.

Mode: single, disjoint, mixed, batch.

This layer expects dense inputs when working in batch mode.

This layer computes a convolution similar to layers.GraphConv, but uses the attention mechanism to weight the adjacency matrix instead of using the normalized Laplacian: $$\X' = \mathbf{\alpha}\X\W + \b$$ where $$\mathbf{\alpha}_{ij} =\frac{ \exp\left(\mathrm{LeakyReLU}\left( \a^{\top} [(\X\W)_i \, \| \, (\X\W)_j]\right)\right)}{\sum\limits_{k \in \mathcal{N}(i) \cup \{ i \}} \exp\left(\mathrm{LeakyReLU}\left( \a^{\top} [(\X\W)_i \, \| \, (\X\W)_k]\right)\right)}$$ where $\a \in \mathbb{R}^{2F'}$ is a trainable attention kernel. Dropout is also applied to $\alpha$ before computing $\Z$. Parallel attention heads are computed in parallel and their results are aggregated by concatenation or average.

Input

• Node features of shape ([batch], n_nodes, n_node_features);
• Binary adjacency matrix of shape ([batch], n_nodes, n_nodes);

Output

• Node features with the same shape as the input, but with the last dimension changed to channels;
• if return_attn_coef=True, a list with the attention coefficients for each attention head. Each attention coefficient matrix has shape ([batch], n_nodes, n_nodes).

Arguments

• channels: number of output channels;
• attn_heads: number of attention heads to use;
• concat_heads: bool, whether to concatenate the output of the attention heads instead of averaging;
• dropout_rate: internal dropout rate for attention coefficients;
• return_attn_coef: if True, return the attention coefficients for the given input (one n_nodes x n_nodes matrix for each head).
• add_self_loops: if True, add self loops to the adjacency matrix.
• activation: activation function;
• use_bias: bool, add a bias vector to the output;
• kernel_initializer: initializer for the weights;
• attn_kernel_initializer: initializer for the attention weights;
• bias_initializer: initializer for the bias vector;
• kernel_regularizer: regularization applied to the weights;
• attn_kernel_regularizer: regularization applied to the attention kernels;
• bias_regularizer: regularization applied to the bias vector;
• activity_regularizer: regularization applied to the output;
• kernel_constraint: constraint applied to the weights;
• attn_kernel_constraint: constraint applied to the attention kernels;
• bias_constraint: constraint applied to the bias vector.

---

[source]

GatedGraphConv

spektral.layers.GatedGraphConv(channels, n_layers, activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)

A gated graph convolutional layer from the paper

Gated Graph Sequence Neural Networks
Yujia Li et al.

Mode: single, disjoint, mixed.

This layer expects a sparse adjacency matrix.

This layer computes $\x_i' = \h^{(L)}_i$ where: $$\begin{align} & \h^{(0)}_i = \x_i \| \mathbf{0} \\ & \m^{(l)}_i = \sum\limits_{j \in \mathcal{N}(i)} \h^{(l - 1)}_j \W \\ & \h^{(l)}_i = \textrm{GRU} \left(\m^{(l)}_i, \h^{(l - 1)}_i \right) \\ \end{align}$$ where $\textrm{GRU}$ is a gated recurrent unit cell.

Input

• Node features of shape (n_nodes, n_node_features); note that n_node_features must be smaller or equal than channels.
• Binary adjacency matrix of shape (n_nodes, n_nodes).

Output

• Node features with the same shape of the input, but the last dimension changed to channels.

Arguments

• channels: integer, number of output channels;
• n_layers: integer, number of iterations with the GRU cell;
• activation: activation function;
• use_bias: bool, add a bias vector to the output;
• kernel_initializer: initializer for the weights;
• bias_initializer: initializer for the bias vector;
• kernel_regularizer: regularization applied to the weights;
• bias_regularizer: regularization applied to the bias vector;
• activity_regularizer: regularization applied to the output;
• kernel_constraint: constraint applied to the weights;
• bias_constraint: constraint applied to the bias vector.

---

[source]

GCNConv

spektral.layers.GCNConv(channels, activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)

A graph convolutional layer (GCN) from the paper

Semi-Supervised Classification with Graph Convolutional Networks
Thomas N. Kipf and Max Welling

Mode: single, disjoint, mixed, batch.

This layer computes: $$\X' = \hat \D^{-1/2} \hat \A \hat \D^{-1/2} \X \W + \b$$ where $\hat \A = \A + \I$ is the adjacency matrix with added self-loops and $\hat\D$ is its degree matrix.

Input

• Node features of shape ([batch], n_nodes, n_node_features);
• Modified Laplacian of shape ([batch], n_nodes, n_nodes); can be computed with spektral.utils.convolution.gcn_filter.

Output

• Node features with the same shape as the input, but with the last dimension changed to channels.

Arguments

• channels: number of output channels;
• activation: activation function;
• use_bias: bool, add a bias vector to the output;
• kernel_initializer: initializer for the weights;
• bias_initializer: initializer for the bias vector;
• kernel_regularizer: regularization applied to the weights;
• bias_regularizer: regularization applied to the bias vector;
• activity_regularizer: regularization applied to the output;
• kernel_constraint: constraint applied to the weights;
• bias_constraint: constraint applied to the bias vector.

---

[source]

GeneralConv

spektral.layers.GeneralConv(channels=256, batch_norm=True, dropout=0.0, aggregate='sum', activation='prelu', use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)

A general convolutional layer from the paper

Design Space for Graph Neural Networks
Jiaxuan You et al.

Mode: single, disjoint, mixed.

This layer expects a sparse adjacency matrix.

This layer computes: $$\x_i' = \mathrm{Agg} \left( \left\{ \mathrm{Act} \left( \mathrm{Dropout} \left( \mathrm{BN} \left( \x_j \W + \b \right) \right) \right), j \in \mathcal{N}(i) \right\} \right)$$

where $\mathrm{Agg}$ is an aggregation function for the messages, $\mathrm{Act}$ is an activation function, $\mathrm{Dropout}$ applies dropout to the node features, and $\mathrm{BN}$ applies batch normalization to the node features.

This layer supports the PReLU activation via the 'prelu' keyword.

The default parameters of this layer are selected according to the best results obtained in the paper, and should provide a good performance on many node-level and graph-level tasks, without modifications. The defaults are as follows:

• 256 channels
• Batch normalization
• No dropout
• PReLU activation
• Sum aggregation

If you are uncertain about which layers to use for your GNN, this is a safe choice. Check out the original paper for more specific configurations.

Input

• Node features of shape (n_nodes, n_node_features);
• Binary adjacency matrix of shape (n_nodes, n_nodes).

Output

• Node features with the same shape of the input, but the last dimension changed to channels.

Arguments

• channels: integer, number of output channels;
• batch_norm: bool, whether to use batch normalization;
• dropout: float, dropout rate;
• aggregate: string or callable, an aggregation function. Supported aggregations: 'sum', 'mean', 'max', 'min', 'prod'.
• activation: activation function. This layer also supports the advanced activation PReLU by passing activation='prelu'.
• use_bias: bool, add a bias vector to the output;
• kernel_initializer: initializer for the weights;
• bias_initializer: initializer for the bias vector;
• kernel_regularizer: regularization applied to the weights;
• bias_regularizer: regularization applied to the bias vector;
• activity_regularizer: regularization applied to the output;
• kernel_constraint: constraint applied to the weights;
• bias_constraint: constraint applied to the bias vector.

---

[source]

GCSConv

spektral.layers.GCSConv(channels, activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)

A GraphConv layer with a trainable skip connection.

Mode: single, disjoint, mixed, batch.

This layer computes: $$\Z' = \D^{-1/2} \A \D^{-1/2} \X \W_1 + \X \W_2 + \b$$ where $\A$ does not have self-loops.

Input

• Node features of shape ([batch], n_nodes, n_node_features);
• Normalized adjacency matrix of shape ([batch], n_nodes, n_nodes); can be computed with spektral.utils.convolution.normalized_adjacency.

Output

• Node features with the same shape as the input, but with the last dimension changed to channels.

Arguments

• channels: number of output channels;
• activation: activation function;
• use_bias: bool, add a bias vector to the output;
• kernel_initializer: initializer for the weights;
• bias_initializer: initializer for the bias vector;
• kernel_regularizer: regularization applied to the weights;
• bias_regularizer: regularization applied to the bias vector;
• activity_regularizer: regularization applied to the output;
• kernel_constraint: constraint applied to the weights;
• bias_constraint: constraint applied to the bias vector.

---

[source]

GINConv

spektral.layers.GINConv(channels, epsilon=None, mlp_hidden=None, mlp_activation='relu', mlp_batchnorm=True, aggregate='sum', activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)

A Graph Isomorphism Network (GIN) from the paper

How Powerful are Graph Neural Networks?
Keyulu Xu et al.

Mode: single, disjoint, mixed.

This layer expects a sparse adjacency matrix.

This layer computes for each node $i$: $$\x_i' = \textrm{MLP}\big( (1 + \epsilon) \cdot \x_i + \sum\limits_{j \in \mathcal{N}(i)} \x_j \big)$$ where $\textrm{MLP}$ is a multi-layer perceptron.

Input

• Node features of shape (n_nodes, n_node_features);
• Binary adjacency matrix of shape (n_nodes, n_nodes).

Output

• Node features with the same shape of the input, but the last dimension changed to channels.

Arguments

• channels: integer, number of output channels;
• epsilon: unnamed parameter, see the original paper and the equation above. By setting epsilon=None, the parameter will be learned (default behaviour). If given as a value, the parameter will stay fixed.
• mlp_hidden: list of integers, number of hidden units for each hidden layer in the MLP (if None, the MLP has only the output layer);
• mlp_activation: activation for the MLP layers;
• mlp_batchnorm: apply batch normalization after every hidden layer of the MLP;
• activation: activation function;
• use_bias: bool, add a bias vector to the output;
• kernel_initializer: initializer for the weights;
• bias_initializer: initializer for the bias vector;
• kernel_regularizer: regularization applied to the weights;
• bias_regularizer: regularization applied to the bias vector;
• activity_regularizer: regularization applied to the output;
• kernel_constraint: constraint applied to the weights;
• bias_constraint: constraint applied to the bias vector.

---

[source]

GraphSageConv

spektral.layers.GraphSageConv(channels, aggregate='mean', activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)

A GraphSAGE layer from the paper

Inductive Representation Learning on Large Graphs
William L. Hamilton et al.

Mode: single, disjoint, mixed.

This layer expects a sparse adjacency matrix.

This layer computes: $$\X' = \big[ \textrm{AGGREGATE}(\X) \| \X \big] \W + \b; \\ \X' = \frac{\X'}{\|\X'\|}$$ where $\textrm{AGGREGATE}$ is a function to aggregate a node's neighbourhood. The supported aggregation methods are: sum, mean, max, min, and product.

Input

• Node features of shape (n_nodes, n_node_features);
• Binary adjacency matrix of shape (n_nodes, n_nodes).

Output

• Node features with the same shape as the input, but with the last dimension changed to channels.

Arguments

• channels: number of output channels;
• aggregate_op: str, aggregation method to use ('sum', 'mean', 'max', 'min', 'prod');
• activation: activation function;
• use_bias: bool, add a bias vector to the output;
• kernel_initializer: initializer for the weights;
• bias_initializer: initializer for the bias vector;
• kernel_regularizer: regularization applied to the weights;
• bias_regularizer: regularization applied to the bias vector;
• activity_regularizer: regularization applied to the output;
• kernel_constraint: constraint applied to the weights;
• bias_constraint: constraint applied to the bias vector.

---

[source]

GTVConv

spektral.layers.GTVConv(channels, delta_coeff=1.0, epsilon=0.001, activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)

A graph total variation convolutional layer (GTVConv) from the paper

Total Variation Graph Neural Networks
Jonas Berg Hansen and Filippo Maria Bianchi

Mode: single, disjoint, batch.

This layer computes $$\X' = \sigma\left[\left(\I - \delta\L_\hat{\mathbf{\Gamma}}\right) \tilde{\X} \right]$$ where $$\begin{align} \tilde{\X} &= \X \W\\[5pt] \L_\hat{\mathbf{\Gamma}} &= \D_\mathbf{\hat{\Gamma}} - \hat{\mathbf{\Gamma}}\\[5pt] [\hat{\mathbf{\Gamma}}]_{ij} &= \frac{[\mathbf{A}]_{ij}}{\max\{||\tilde{\x}_i-\tilde{\x}_j||_1, \epsilon\}}\\ \end{align}$$

Input

• Node features of shape (batch, n_nodes, n_node_features);
• Adjacency matrix of shape (batch, n_nodes, n_nodes);

Output

• Node features with the same shape as the input, but with the last dimension changed to channels.

Arguments

• channels: number of output channels;
• delta_coeff: step size for gradient descent of GTV
• epsilon: small number used to numerically stabilize the computation of new adjacency weights
• activation: activation function;
• use_bias: bool, add a bias vector to the output;
• kernel_initializer: initializer for the weights;
• bias_initializer: initializer for the bias vector;
• kernel_regularizer: regularization applied to the weights;
• bias_regularizer: regularization applied to the bias vector;
• activity_regularizer: regularization applied to the output;
• kernel_constraint: constraint applied to the weights;
• bias_constraint: constraint applied to the bias vector.

---

[source]

TAGConv

spektral.layers.TAGConv(channels, K=3, aggregate='sum', activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)

A Topology Adaptive Graph Convolutional layer (TAG) from the paper

Topology Adaptive Graph Convolutional Networks
Jian Du et al.

Mode: single, disjoint, mixed.

This layer expects a sparse adjacency matrix.

This layer computes: $$\Z = \sum\limits_{k=0}^{K} \D^{-1/2}\A^k\D^{-1/2}\X\W^{(k)}$$

Input

• Node features of shape (n_nodes, n_node_features);
• Binary adjacency matrix of shape (n_nodes, n_nodes).

Output

• Node features with the same shape of the input, but the last dimension changed to channels.

Arguments

• channels: integer, number of output channels;
• K: the order of the layer (i.e., the layer will consider a K-hop neighbourhood for each node);
• activation: activation function;
• use_bias: bool, add a bias vector to the output;
• kernel_initializer: initializer for the weights;
• bias_initializer: initializer for the bias vector;
• kernel_regularizer: regularization applied to the weights;
• bias_regularizer: regularization applied to the bias vector;
• activity_regularizer: regularization applied to the output;
• kernel_constraint: constraint applied to the weights;
• bias_constraint: constraint applied to the bias vector.

---

[source]

XENetConv

spektral.layers.XENetConv(stack_channels, node_channels, edge_channels, attention=True, node_activation=None, edge_activation=None, aggregate='sum', use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)

A XENet convolutional layer from the paper

XENet: Using a new graph convolution to accelerate the timeline for protein design on quantum computers
Jack B. Maguire, Daniele Grattarola, Eugene Klyshko, Vikram Khipple Mulligan, Hans Melo

Mode: single, disjoint, mixed.

This layer expects a sparse adjacency matrix.

For a version of this layer that supports batch mode, you can use spektral.layers.XENetDenseConv as a drop-in replacement.

This layer computes for each node $i$: $$\s_{ij} = \text{PReLU} \left( (\x_{i} \| \x_{j} \| \e_{ij} \| \e_{ji}) \W^{(s)} + \b^{(s)} \right) \\ \s^{(\text{out})}_{i} = \sum\limits_{j \in \mathcal{N}(i)} \s_{ij} \\ \s^{(\text{in})}_{i} = \sum\limits_{j \in \mathcal{N}(i)} \s_{ji} \\ \x_{i}' = \sigma\left( (\x_{i} \| \s^{(\text{out})}_{i} \| \s^{(\text{in})}_{i}) \W^{(n)} + \b^{(n)} \right) \\ \e_{ij}' = \sigma\left( \s_{ij} \W^{(e)} + \b^{(e)} \right)$$

Input

• Node features of shape ([batch], n_nodes, n_node_features);
• Binary adjacency matrices of shape ([batch], n_nodes, n_nodes);
• Edge features of shape (num_edges, n_edge_features);

Output

• Node features with the same shape of the input, but the last dimension changed to node_channels.
• Edge features with the same shape of the input, but the last dimension changed to edge_channels.

Arguments

• stack_channels: integer or list of integers, number of channels for the hidden layers;
• node_channels: integer, number of output channels for the nodes;
• edge_channels: integer, number of output channels for the edges;
• attention: whether to use attention when aggregating the stacks;
• node_activation: activation function for nodes;
• edge_activation: activation function for edges;
• use_bias: bool, add a bias vector to the output;
• kernel_initializer: initializer for the weights;
• bias_initializer: initializer for the bias vector;
• kernel_regularizer: regularization applied to the weights;
• bias_regularizer: regularization applied to the bias vector;
• activity_regularizer: regularization applied to the output;
• kernel_constraint: constraint applied to the weights;
• bias_constraint: constraint applied to the bias vector.

---

[source]

GINConvBatch

spektral.layers.GINConvBatch(channels, epsilon=None, mlp_hidden=None, mlp_activation='relu', mlp_batchnorm=True, aggregate='sum', activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)

A batch-mode version of GINConv.

Mode: batch.

This layer expects a dense adjacency matrix.

---

[source]

XENetConvBatch

spektral.layers.XENetConvBatch(stack_channels, node_channels, edge_channels, attention=True, node_activation=None, edge_activation=None, aggregate='sum', use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)

A batch-mode version of XENetConv.

Mode: batch.

This layer expects a dense adjacency matrix.