How do I implement IEnumerable<T>

If you choose to use a generic collection, such as List<MyObject> instead of ArrayList, you'll find that the List<MyObject> will provide both generic and non-generic enumerators that you can use.

using System.Collections;

class MyObjects : IEnumerable<MyObject>
{
    List<MyObject> mylist = new List<MyObject>();

    public MyObject this[int index]  
    {  
        get { return mylist[index]; }  
        set { mylist.Insert(index, value); }  
    } 

    public IEnumerator<MyObject> GetEnumerator()
    {
        return mylist.GetEnumerator();
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return this.GetEnumerator();
    }
}

You probably do not want an explicit implementation of IEnumerable<T> (which is what you've shown).

The usual pattern is to use IEnumerable<T>'s GetEnumerator in the explicit implementation of IEnumerable:

class FooCollection : IEnumerable<Foo>, IEnumerable
{
    SomeCollection<Foo> foos;

    // Explicit for IEnumerable because weakly typed collections are Bad
    System.Collections.IEnumerator IEnumerable.GetEnumerator()
    {
        // uses the strongly typed IEnumerable<T> implementation
        return this.GetEnumerator();
    }

    // Normal implementation for IEnumerable<T>
    IEnumerator<Foo> GetEnumerator()
    {
        foreach (Foo foo in this.foos)
        {
            yield return foo;
            //nb: if SomeCollection is not strongly-typed use a cast:
            // yield return (Foo)foo;
            // Or better yet, switch to an internal collection which is
            // strongly-typed. Such as List<T> or T[], your choice.
        }

        // or, as pointed out: return this.foos.GetEnumerator();
    }
}

Why do you do it manually? yield return automates the entire process of handling iterators. (I also wrote about it on my blog, including a look at the compiler generated code).

If you really want to do it yourself, you have to return a generic enumerator too. You won't be able to use an ArrayList any more since that's non-generic. Change it to a List<MyObject> instead. That of course assumes that you only have objects of type MyObject (or derived types) in your collection.