Classification with Softmax Outputs

The branch outputs then meet at a Keras merge operation, such as concatenate or add.

Programming

From Separate Data to One Answer

A single-input model receives one stream of data. A multi-input model receives several streams, sends each stream through its own path, and combines the resulting representations before producing an output. This is why a multi-input model should not be viewed as one straight sequence of layers. Its structure is a graph with branches that eventually meet.

In the real world

A question-answering system can use one input for a question and another input for a text snippet containing information needed to answer it. The question and text travel through separate branches before their representations are combined.

How does each input move through its own encoder branch before the branch representations are combined?

The Two-Input Graph

The Keras functional API is designed for graph-shaped models. A two-input model begins with two input nodes. Each input is passed through its own branch encoder. The branch outputs then meet at a Keras merge operation, such as concatenate or add. The merged representation is passed to an output layer that produces the answer tensor.

Key point

The names text and question identify the two input nodes. They are part of the model structure, not decorative labels, because the names can later be used to map dictionary-based training data to the correct inputs.

python

text_input = keras.Input(name="text")
question_input = keras.Input(name="question")

text_representation = text_encoder(text_input)
question_representation = question_encoder(question_input)

merged = keras.layers.concatenate(
    [text_representation, question_representation]
)
answer = output_layer(merged)

model = keras.Model(
    inputs=[text_input, question_input],
    outputs=answer
)

text_input = keras.Input(name="text")
question_input = keras.Input(name="question")

text_representation = text_encoder(text_input)
question_representation = question_encoder(question_input)

merged = keras.layers.concatenate(
    [text_representation, question_representation]
)
answer = output_layer(merged)

model = keras.Model(
    inputs=[text_input, question_input],
    outputs=answer
)

This code is a structural example. The important path is explicit: the text input enters the text branch, the question input enters the question branch, both branch representations reach the merge operation, and the output layer receives the merged result. The Model construction receives both input tensors and produces the answer tensor.

What are the main stages of the model, and how are the input layers, encoders, merge operation, and output layer connected?

Choosing the Merge Operation

Merge strategy	What it does	What to inspect
Concatenate	Combines the branch representations into one merged representation	Whether both branch outputs are being brought into the merged result
Add	Combines the branch representations through an addition operation	Whether addition is the intended way to combine the branch outputs

Keras provides concatenate and add as examples of operations for bringing branch representations together.

Concatenation and addition are not interchangeable descriptions of the graph. They are different merge operations, and the choice determines how the branch representations are brought together. When a multi-input model behaves unexpectedly, inspect the merge line instead of assuming that every merge combines information in the same way.

What changes in the merged representation when branch outputs are concatenated instead of added?

Supplying Training Data

Training data must preserve the model's input structure. The source describes two supported forms. A list supplies arrays in the same order as the model's inputs. A dictionary supplies arrays by input name. Dictionary-based training requires named inputs.

python

model.fit(
    [text_array, question_array],
    answer_array
)

model.fit(
    {
        "text": text_array,
        "question": question_array
    },
    answer_array
)

model.fit(
    [text_array, question_array],
    answer_array
)

model.fit(
    {
        "text": text_array,
        "question": question_array
    },
    answer_array
)

In the list form, position carries the meaning: the first array belongs to the first model input, and the second array belongs to the second model input. In the dictionary form, the names carry the meaning: the array under text is associated with the input named text, and the array under question is associated with the input named question.

How does each array in a list or dictionary connect to the correct input layer?

Tracing a Failure

A multi-input model can fail at several distinct boundaries. Treat the model as a trace: data enters an input node, passes through its branch, reaches the merge operation, and contributes to the output. The most useful debugging question is where the actual data path first differs from the intended graph.

Common mistake
Checking only the final classifier when the real problem is earlier in the graph.
What it looks like
The output is wrong, but the text array may have entered the question branch.
Why it fails
The final output depends on the paths and merge operation that came before it.
Do this instead
Inspect the input branches first, then the merge stage, and only then the final classifier.
Common mistake
Supplying list-based training arrays in an order different from the model's input order.
What it looks like
The list places question_array first even though the model's first input is text.
Why it fails
List-based training maps arrays by position.
Do this instead
Compare the list order with the model's declared input order before training.
Common mistake
Using dictionary-based training with names that do not identify the model inputs.
What it looks like
The data dictionary uses keys that do not correspond to the named text and question inputs.
Why it fails
Dictionary-based training relies on input names.
Do this instead
Use the model's input names as the dictionary keys.
Common mistake
Assuming concatenate and add represent the same merge behavior.
What it looks like
The graph uses add when the intended design requires concatenate.
Why it fails
The merge operation determines how the branch representations are brought together.
Do this instead
Inspect the merge line and verify that it uses the intended Keras operation.

How can a failure be isolated to an input branch, the merge stage, or the training-data mapping?

Practice the Trace

MEDIUM

A model has two named inputs, text and question. Its input list is ordered as [text, question]. You have text_array and question_array. Explain how you would supply the data using the list form and the dictionary form. Then identify the first three places you would inspect if the model produced an unexpected output.

Hints

For the list form, use the model's input order.
For the dictionary form, use the input names.
Trace the path from data mapping to input branches to the merge operation.

Your attempt (not saved)

Worked example

Checking a Two-Input Training Call

The model inputs are ordered as text followed by question. Decide whether each training-data form preserves the intended mapping.

1
List form
[text_array, question_array] preserves the declared input order, so the text array maps to text and the question array maps to question.
2
Dictionary form
{"text": text_array, "question": question_array} uses the named inputs, so each array is associated with its matching input name.
3
Debugging check
If the result is unexpected, inspect the data mapping first, then verify each branch receives its intended data, and then inspect whether the merge operation is concatenate or add as intended.

Result

Both forms preserve the intended mapping when the list order is correct and the dictionary keys match the named model inputs.

Key Takeaways

1
A multi-input Keras functional model gives each data source its own branch before combining the branch representations.
2
A two-input graph contains input layers, branch encoders, a merge operation, and an output layer.
3
Concatenate and add are different Keras merge operations, so the merge line deserves deliberate inspection.
4
List-based training data maps by model input order, while dictionary-based training data maps by named inputs.
5
When debugging, trace the actual data path from training-data mapping to input branches to the merge stage and then to the output.

At a glance

Separate inputs travel through independent branches in a Keras functional model.
The branches meet at a merge operation such as concatenate or add before reaching the output layer.
Input names and input order determine how training arrays are connected to the model.
A reliable debugging strategy is to find where the actual data path first differs from the intended graph.

Related concepts

More concepts starting with C