Let I be a finite set of integers and F be a finite set of maps of the form n → ki n + i with integer coefficients. For an integer base k ≥ 2, we study the k-recognizability of the minimal set X of integers containing I and satisfying ϕ(X) ⊆ X for all ϕ ∈ F. In particular, solving a conjecture of Allouche, Shallit and Skordev, we show under some technical conditions that if two of the constants ki are multiplicatively independent, then X is not k-recognizable for any k ≥ 2.