Remove duplicates in list (Prolog)

A simpler (and likely faster) solution is to use library predicate sort/2 which remove duplicates in O(n log n). Definitely works in Yap prolog and SWIPL

You are getting multiple solutions due to Prolog's backtracking. Technically, each solution provided is correct, which is why it is being generated. If you want just one solution to be generated, you are going to have to stop backtracking at some point. This is what the Prolog cut is used for. You might find that reading up on that will help you with this problem.

Update: Right. Your member() predicate is evaluating as true in several different ways if the first variable is in multiple positions in the second variable.

I've used the name mymember() for this predicate, so as not to conflict with GNU Prolog's builtin member() predicate. My knowledge base now looks like this:

mymember(X,[X|_]).
mymember(X,[_|T]) :- mymember(X,T).

not(A) :- \+ call(A).

set([],[]).
set([H|T],[H|Out]) :-
    not(mymember(H,T)),
    set(T,Out).
set([H|T],Out) :-
    mymember(H,T),
    set(T,Out).

So, mymember(1, [1, 1, 1]). evaluates as true in three different ways:

| ?- mymember(1, [1, 1, 1]).

true ? a

true

true

no

If you want to have only one answer, you're going to have to use a cut. Changing the first definition of mymember() to this:

mymember(X,[X|_]) :- !.

Solves your problem.

Furthermore, you can avoid not() altogether, if you wish, by defining a notamember() predicate yourself. The choice is yours.

You are on the right track... Stay pure---it's easy!

Use reified equality predicates =/3 and dif/3 in combination with if_/3, as implemented in Prolog union for A U B U C:

=(X, Y, R) :- X == Y,    !, R = true.
=(X, Y, R) :- ?=(X, Y),  !, R = false. % syntactically different
=(X, Y, R) :- X \= Y,    !, R = false. % semantically different
=(X, Y, R) :- R == true, !, X = Y.
=(X, X, true).
=(X, Y, false) :-
   dif(X, Y).

% dif/3 is defined like (=)/3
dif(X, Y, R) :- X == Y,    !, R = false.
dif(X, Y, R) :- ?=(X, Y),  !, R = true. % syntactically different
dif(X, Y, R) :- X \= Y,    !, R = true. % semantically different
dif(X, Y, R) :- R == true, !, X \= Y.
dif(X, Y, true) :-         % succeed first!
   dif(X, Y).
dif(X, X, false).

if_(C_1, Then_0, Else_0) :-
   call(C_1, Truth),
   functor(Truth,_,0),  % safety check
   ( Truth == true -> Then_0 ; Truth == false, Else_0 ).

Based on these predicates we build a reified membership predicate list_item_isMember/3. It is semantically equivalent with memberd_truth/3 by @false. We rearrange the argument order so the list is the 1st argument. This enables first-argument indexing which prevents leaving useless choice-points behind as memberd_truth/3 would create.

list_item_isMember([],_,false).
list_item_isMember([X|Xs],E,Truth) :-
   if_(E = X, Truth = true, list_item_isMember(Xs,E,Truth)).

list_set([],[]).
list_set([X|Xs],Ys) :-
    if_(list_item_isMember(Xs,X), Ys = Ys0, Ys = [X|Ys0]),
    list_set(Xs,Ys0).

A simple query shows that all redundant answers have been eliminated and that the goal succeeds without leaving any choice-points behind:

?- list_set([1,2,3,4,1,2,3,4,1,2,3,1,2,1],Xs).
Xs = [4,3,2,1].                         % succeeds deterministically

---

Edit 2015-04-23

I was inspired by @Ludwig's answer of set/2, which goes like this:

set([],[]).
set([H|T],[H|T1]) :- subtract(T,[H],T2), set(T2,T1).

SWI-Prolog's builtin predicate subtract/3 can be non-monotone, which may restrict its use. list_item_subtracted/3 is a monotone variant of it:

list_item_subtracted([],_,[]).
list_item_subtracted([A|As],E,Bs1) :-
    if_(dif(A,E), Bs1 = [A|Bs], Bs = Bs1),
    list_item_subtracted(As,E,Bs).

list_setB/2 is like set/2, but is based on list_item_subtracted/3---not subtract/3:

list_setB([],[]).
list_setB([X|Xs1],[X|Ys]) :-
    list_item_subtracted(Xs1,X,Xs),
    list_setB(Xs,Ys).

The following queries compare list_set/2 and list_setB/2:

?- list_set([1,2,3,4,1,2,3,4,1,2,3,1,2,1], Xs).
Xs = [4,3,2,1].                          % succeeds deterministically
?- list_setB([1,2,3,4,1,2,3,4,1,2,3,1,2,1],Xs).
Xs = [1,2,3,4].                          % succeeds deterministically

?- list_set(Xs,[a,b]).
   Xs = [a,b]
;  Xs = [a,b,b]
;  Xs = [a,b,b,b] 
...                                      % does not terminate universally
?- list_setB(Xs,[a,b]).
   Xs = [a,b]
;  Xs = [a,b,b]
;  Xs = [a,b,b,b]
...                                      % does not terminate universally