Since your data have a header, the as.matrix() function is converting numerical values to characters.
You can use the sapply() function to make data frame values numeric and then convert the data frame to the data matrix.
Apply the highlighted change to your code, and it should work without any error:
library(mlbench)
data("BreastCancer")
# generate data
X <- subset(BreastCancer, select = -c(Class,Id)) # remove Id and Class
X <- as.matrix(sapply(X, as.numeric)) # keep data as numeric