How can I make a trainable parameter in keras?

thanks for looking my question.

For example.

The final output is the sum of two matrix A and B,like this:

output = keras.layers.add([A, B])


Now,I want to build a new parameter x to change the output.

I want to make newoutput = Ax+B(1-x)

and x is a trainable parameter in my network.

what should I do?
please help me ~ thanks very much!

edit(part of code ):

conv1 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(input)
drop1 = Dropout(0.5)(conv1)
pool1 = MaxPooling2D(pool_size=(2, 2))(drop1)

conv2 = Conv2D(1024, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool1)
conv2 = Conv2D(1024, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv2)
drop2 = Dropout(0.5)(conv2)

up1 = Conv2D(512, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(drop2))

#the line I want to change:
merge = add([drop2,up1])
#this layer is simply add drop2 and up1 layer.now I want to add a trainable parameter x to adjust the weight of thoese two layers.


I tried to use the codes,but still occured some questions:

1.how can I use my own layer?

merge = Mylayer()(drop2,up1)


or otherway?

2.what is the meaning of out_dim?
those parameters are all 3-dim matrix.what is the mening of out_dim?

thank you...T.T

edit2(solved)

from keras import backend as K
from keras.engine.topology import Layer
import numpy as np

from keras.layers import add

class MyLayer(Layer):

def __init__(self, **kwargs):
super(MyLayer, self).__init__(**kwargs)

def build(self, input_shape):

self._x = K.variable(0.5)
self.trainable_weights = [self._x]

super(MyLayer, self).build(input_shape) # Be sure to call this at the end

def call(self, x):
A, B = x
result = add([self._x*A ,(1-self._x)*B])
return result

def compute_output_shape(self, input_shape):
return input_shape[0]


1 Answer
1

You have to create a custom class which inherits from Layer and create the trainable parameter using self.add_weight(...). You can find an example of this here and there.

Layer

self.add_weight(...)

For your example, the layer would somehow look like this:

from keras import backend as K
from keras.engine.topology import Layer
import numpy as np

class MyLayer(Layer):

def __init__(self, output_dim, **kwargs):
self.output_dim = output_dim
super(MyLayer, self).__init__(**kwargs)

def build(self, input_shape):
# Create a trainable weight variable for this layer.
self._A = self.add_weight(name='A',
shape=(input_shape[1], self.output_dim),
initializer='uniform',
trainable=True)
self._B = self.add_weight(name='B',
shape=(input_shape[1], self.output_dim),
initializer='uniform',
trainable=True)
super(MyLayer, self).build(input_shape) # Be sure to call this at the end

def call(self, x):
return K.dot(x, self._A) + K.dot(1-x, self._B)

def compute_output_shape(self, input_shape):
return (input_shape[0], self.output_dim)


Edit: Just based on the names I (wrongly) assumed that x is the layers input and you want to optimize A and B. But, as you stated, you want to optimize x. For this, you can do something like this:

x

A

B

x

from keras import backend as K
from keras.engine.topology import Layer
import numpy as np

class MyLayer(Layer):

def __init__(self, **kwargs):
super(MyLayer, self).__init__(**kwargs)

def build(self, input_shape):
# Create a trainable weight variable for this layer.
self._x = self.add_weight(name='x',
shape=(1,),
initializer='uniform',
trainable=True)
super(MyLayer, self).build(input_shape) # Be sure to call this at the end

def call(self, x):
A, B = x
return K.dot(self._x, A) + K.dot(1-self._x, B)

def compute_output_shape(self, input_shape):
return input_shape[0]


Edit2: You can call this layer using

merge = Mylayer()([drop2,up1])


x

Required, but never shown

Required, but never shown

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.