ORA-01799: a column may not be outer-joined to a subquery

you can rewrite that by pushing the sub query so that its not outer joined:

select Count(C.setid)
  from mycustomer C
       left outer join (select *
                          from mycustoption co
                         where co.effdt <= (select Max(COI.effdt)
                                              from mycustoption COI
                                             where COI.setid = co.setid
                                               and COI.cust_id = co.cust_id
                                               and COI.effdt <= sysdate)) co
                    on ( C.setid = CO.setid
                         and C.cust_id = CO.cust_id ) 

Your question has already been answered, but someone might have a slightly different case where they need to get the latest EFFDT based on a column, instead of a fixed date. For those cases, I only found one IMPERFECT option, and one UGLY solution...

Imperfect option:

SELECT ...
FROM MYTABLE N, CUST_OPT C
WHERE  etc...
AND C.SETID           (+) = N.SETID
AND C.CUST_ID         (+) = N.CUST_ID
AND NVL(C.EFFDT,TO_DATE('01011900','DDMMYYYY')) = NVL((SELECT MAX(EFFDT)
                                                       FROM CUST_OPT SC
                                                       WHERE SC.SETID = C.SETID
                                                       AND   SC.CUST_ID = C.CUST_ID
                                                       AND   SC.EFFDT <= N.ISSUE_DT)
                                                       ,TO_DATE('01011900','DDMMYYYY'))

It is an imperfect option because if the CUST_OPT table has future dates, but no current (<=N.ISSUE_DT) dates, the outer join will not work and no rows will be returned. In general PeopleSoft terms (yes I saw your SETID+EFFDT there! ;-D) this wouldn't happen very often as people tend to create one 01/01/1900 EFFDT to make a first value effective since "forever", but since it's not always the case; we also have an ugly solution:

I also found one UGLY option (but I actually recommend it, and it solves the problem, so let's call it a solution), which is this:

SELECT n.field1, n.field2,
       CASE WHEN NVL(c.EFFDT,n.ISSUE_DT-1)<=n.ISSUE_DT THEN c.field1 ELSE NULL END,
       CASE WHEN NVL(c.EFFDT,n.ISSUE_DT-1)<=n.ISSUE_DT THEN c.field2 ELSE NULL END
FROM MYTABLE N, CUST_OPT C
WHERE  etc...
AND C.SETID           (+) = N.SETID
AND C.CUST_ID         (+) = N.CUST_ID
AND NVL(C.EFFDT,TO_DATE('01011900','DDMMYYYY')) = NVL((SELECT MAX(EFFDT)
                                                       FROM CUST_OPT SC
                                                       WHERE SC.SETID = C.SETID
                                                       AND   SC.CUST_ID = C.CUST_ID
                                                       AND   SC.EFFDT <= N.ISSUE_DT)
                                                     ,NVL( (SELECT MIN(EFFDT)
                                                            FROM CUST_OPT SC
                                                            WHERE SC.SETID = C.SETID
                                                            AND   SC.CUST_ID = C.CUST_ID
                                                            AND   SC.EFFDT >= N.ISSUE_DT)
                                                         ,TO_DATE('01011900','DDMMYYYY')
                                                         )
                                                     )

This option WILL return FUTURE rows which must be ignored! So we add the conditions on the SELECT statement that will IGNORE the returned values, if they weren't meant to be retrieved. Like I said... it's an UGLY solution, but it is a solution.

For my ugly solution, if the rows will be processed later in an Application Engine or PL/SQL or whatever; you can, instead of having a CASE statement for each column, just add a new column that will tell you that you fetched "improper" data and ignore the fields later in your code, based on this column, like this:

CASE WHEN NVL(c.EFFDT,n.ISSUE_DT-1)<=n.ISSUE_DT THEN 'N' ELSE 'Y' END AS IGNORE_CUST_OP_COLS

Well, Oracle apparently doesn't support using a subquery inside the join condition for an outer join. So you need to get rid of the subquery.

The question is, why is it there at all? You have "<=" conditions in two places, so the predicate essentially says "all records whose effective date is no later than the latest effective date that is no later than now". If that's what you really want, you could simplify it to "all records whose effective date is no later than now", i.e.:

ON 
    (C.SETID = CO.SETID 
        AND C.CUST_ID = CO.CUST_ID 
        AND CO.effdt <= SYSDATE    
)

Voila, no subquery.

But is that really what you want, or did you mean that first "<=" to be just "=" -- i.e. find the record with the most recent effective date before now? If that's what you really want, it will be more complex to rewrite.

Option 1

select COUNT(C.SETID)
from MYCUSTOMER C
left outer join (
  select *
  from MYCUSTOPTION CO
    on CO.effdt = (
        select MAX(COI.EFFDT)
        from MYCUSTOPTION COI
        where COI.SETID = CO.SETID
          and COI.CUST_ID = CO.CUST_ID
          and COI.EFFDT <= SYSDATE
        )
  ) CO
  on C.SETID = CO.SETID
    and C.CUST_ID = CO.CUST_ID;

Option 2

select COUNT(C.SETID)
from MYCUSTOMER C
left outer join MYCUSTOPTION CO
  on C.SETID = CO.SETID
    and C.CUST_ID = CO.CUST_ID
where nvl(CO.effdt, to_date('19000101', 'YYYYMMDD')) = NVL((
      select MAX(COI.EFFDT)
      from MYCUSTOPTION COI
      where COI.SETID = CO.SETID
        and COI.CUST_ID = CO.CUST_ID
        and COI.EFFDT <= C.SINCE_DT
      ), to_date('19000101', 'YYYYMMDD'))

Option1 works fine as long as you want to query MYCUSTOPTION table as of "today"(COI.EFFDT <= SYSDATE) which appears to be your req in this case. But it won't work if you want to query MYCUSTOPTION as of "C.SINCE_DT"

Option2 looks bit complex but it works better, allows you to switch between sysdate or any other date field without changing anything else(COI.EFFDT <= C.SINCE_DT)