It is possible to load the weights according to a particular initialization using the attribute kernel_initializer for layers
model.add(Dense(2,kernel_initializer='glorot_normal'))
The default value is glorot_uniform
For initializers where you need to provide arguments
from keras import initializers
model.add(Dense(64, kernel_initializer=initializers.random_normal(stddev=0.01))
As you mentioned Xavier,in Keras, Xavier uniform and Xavier normal are known as glorot uniform and glorot normal respectively.
Refer to Keras Initializers for more initializers
EDIT:
If you want to set the weights using a list of numpy arrays, refer to the this answer