Novelty detection involves identifying novel patterns. They are not usually available during training. Even if they are, the data quantity imbalance leads to a low classification accuracy when a supervised learning scheme is employed. Thus, an unsupervised learning scheme is often employed ignoring those few novel patterns. In this paper, we propose two ways to make use of the few available novel patterns. First, a scheme to determine local thresholds for the Self Organizing Map boundary is proposed. Second, a modification of the Learning Vector Quantization learning rule is proposed so that allows one to keep codebook vectors as far from novel patterns as possible. Experimental results are quite promising.