unitorch.cli.models.visualbert¤
VisualBertProcessor¤
Tip
core/process/visualbert
is the section for configuration of VisualBertProcessor.
Bases: VisualBertProcessor
VisualBERT Processor for text and image inputs.
Initialize the VisualBertProcessor.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
vocab_path |
str
|
The path to the vocabulary file. |
required |
max_seq_length |
Optional[int]
|
The maximum sequence length. Defaults to 128. |
128
|
special_input_ids |
Optional[Dict]
|
A dictionary containing special input IDs. Defaults to an empty dictionary. |
dict()
|
do_lower_case |
Optional[bool]
|
Whether to convert the text to lowercase. Defaults to True. |
True
|
do_basic_tokenize |
Optional[bool]
|
Whether to perform basic tokenization. Defaults to True. |
True
|
do_whole_word_mask |
Optional[bool]
|
Whether to use whole-word masking. Defaults to True. |
True
|
masked_lm_prob |
Optional[float]
|
The probability of masked language model masking. Defaults to 0.15. |
0.15
|
max_predictions_per_seq |
Optional[int]
|
The maximum number of masked language model predictions per sequence. Defaults to 20. |
20
|
Source code in src/unitorch/cli/models/visualbert/processing.py
25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 |
|
from_core_configure
classmethod
¤
from_core_configure(config, **kwargs)
Create an instance of VisualBertProcessor from a core configuration.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
config |
The core configuration. |
required | |
**kwargs |
Additional keyword arguments. |
{}
|
Returns:
Name | Type | Description |
---|---|---|
dict |
A dictionary containing the processor's configuration. |
Source code in src/unitorch/cli/models/visualbert/processing.py
60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 |
|
VisualBertForClassification¤
Tip
core/model/classification/visualbert
is the section for configuration of VisualBertForClassification.
Bases: VisualBertForClassification
VisualBERT for Classification model.
Initialize VisualBertForClassification.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
config_path |
str
|
The path to the model's configuration file. |
required |
num_classes |
Optional[int]
|
The number of classes for classification. Defaults to 1. |
1
|
gradient_checkpointing |
Optional[bool]
|
Whether to use gradient checkpointing to save memory during training. Defaults to False. |
False
|
Source code in src/unitorch/cli/models/visualbert/modeling.py
26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 |
|
forward ¤
forward(
input_ids: Tensor,
attention_mask: Tensor,
token_type_ids: Tensor,
position_ids: Tensor,
visual_embeds: Tensor,
visual_attention_mask: Tensor,
visual_token_type_ids: Tensor,
)
Forward pass of the VisualBertForClassification model.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
input_ids |
Tensor
|
The input token IDs. |
required |
attention_mask |
Tensor
|
The attention mask. |
required |
token_type_ids |
Tensor
|
The token type IDs. |
required |
position_ids |
Tensor
|
The position IDs. |
required |
visual_embeds |
Tensor
|
The visual embeddings. |
required |
visual_attention_mask |
Tensor
|
The visual attention mask. |
required |
visual_token_type_ids |
Tensor
|
The visual token type IDs. |
required |
Returns:
Name | Type | Description |
---|---|---|
ClassificationOutputs |
The output logits of the model. |
Source code in src/unitorch/cli/models/visualbert/modeling.py
86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 |
|
from_core_configure
classmethod
¤
from_core_configure(config, **kwargs)
Create an instance of VisualBertForClassification from a core configuration.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
config |
The core configuration. |
required | |
**kwargs |
Additional keyword arguments. |
{}
|
Returns:
Name | Type | Description |
---|---|---|
VisualBertForClassification |
The initialized VisualBertForClassification instance. |
Source code in src/unitorch/cli/models/visualbert/modeling.py
48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 |
|
VisualBertForPretrain¤
Tip
core/model/pretrain/visualbert
is the section for configuration of VisualBertForPretrain.
Bases: VisualBertForPretrain
VisualBERT for Pretraining model.
Initialize VisualBertForPretrain.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
config_path |
str
|
The path to the model's configuration file. |
required |
gradient_checkpointing |
Optional[bool]
|
Whether to use gradient checkpointing to save memory during training. Defaults to False. |
False
|
Source code in src/unitorch/cli/models/visualbert/modeling.py
128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 |
|
forward ¤
forward(
input_ids: Tensor,
attention_mask: Tensor,
token_type_ids: Tensor,
position_ids: Tensor,
visual_embeds: Tensor,
visual_attention_mask: Tensor,
visual_token_type_ids: Tensor,
nsp_label: Tensor,
mlm_label: Tensor,
mlm_label_mask: Tensor,
)
Forward pass of the VisualBertForPretrain model.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
input_ids |
Tensor
|
The input token IDs. |
required |
attention_mask |
Tensor
|
The attention mask. |
required |
token_type_ids |
Tensor
|
The token type IDs. |
required |
position_ids |
Tensor
|
The position IDs. |
required |
visual_embeds |
Tensor
|
The visual embeddings. |
required |
visual_attention_mask |
Tensor
|
The visual attention mask. |
required |
visual_token_type_ids |
Tensor
|
The visual token type IDs. |
required |
nsp_label |
Tensor
|
The next sentence prediction label. |
required |
mlm_label |
Tensor
|
The masked language model label. |
required |
mlm_label_mask |
Tensor
|
The masked language model label mask. |
required |
Returns:
Name | Type | Description |
---|---|---|
LossOutputs |
The output loss of the model. |
Source code in src/unitorch/cli/models/visualbert/modeling.py
183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 |
|
from_core_configure
classmethod
¤
from_core_configure(config, **kwargs)
Create an instance of VisualBertForPretrain from a core configuration.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
config |
The core configuration. |
required | |
**kwargs |
Additional keyword arguments. |
{}
|
Returns:
Name | Type | Description |
---|---|---|
VisualBertForPretrain |
The initialized VisualBertForPretrain instance. |
Source code in src/unitorch/cli/models/visualbert/modeling.py
146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 |
|