We describe a novel approach that integrates user-generated tags with standard Matrix Factorization to allow users to interactively control recommendations. The tag information is incorporated during the learning phase and relates to the automatically derived latent factors. Thus, the system can change an item’s score whenever the user adjusts a tag’s weight. We implemented a prototype and performed a user study showing that this seems to be a promising way for users to interactively manipulate the set of items recommended based on their user profile or in cold-start situations. Categories and Subject Descriptors H.3.3 [Information Storage and Retrieval]: Information Search and Retrieval—information filtering, search process Keywords Recommender Systems; Interactive Recommending; Matrix Factorization; Tags; User Interfaces