Ayende @ Rahien

It's a girl

Challenge: Robust enumeration over external code

Here is an interesting little problem:

public class Program
{
    private static void Main()
    {
        foreach (int i in RobustEnumerating(Enumerable.Range(0, 10), FaultyFunc))
        {
            Console.WriteLine(i);
        }
    }

    public static IEnumerable<T> RobustEnumerating<T>(
        IEnumerable<T> input,Func<IEnumerable<T>, IEnumerable<T>> func)
    {
        // how to do this?
        return func(input);

    }

    public static IEnumerable<int> FaultyFunc(IEnumerable<int> source)
    {
        foreach (int i in source)
        {
            yield return i/(i%2);
        }
    }
}

This code should not throw, but print:

1
3
5
7
9

Can you make this happen? You can only change the RobustEnumerating method, nothing else in the code

Comments

Juan Lopes
03/04/2010 02:00 AM by
Juan Lopes

public static IEnumerable <t RobustEnumerating <t(

    IEnumerable

<t input,Func <ienumerable<t, IEnumerable func)

{

    foreach(T t in func(input)) {

        try { yield return t; }

        catch { }

    }

}
Ayende Rahien
03/04/2010 02:03 AM by
Ayende Rahien

Juan,

That code doesn't compiles

Michael Stum
03/04/2010 02:21 AM by
Michael Stum

Not tested and ugly and possibly cheating, but:

public static IEnumerable <t RobustEnumerating <t(

    IEnumerable

<t input,Func <ienumerable<t, IEnumerable func)

{


  foreach(T obj in input) {

      try {

          var newList = new List

<t();

          newList.Add(obj);

          yield return func(newList)

      } catch {}

      }

}
Juan Lopes
03/04/2010 02:28 AM by
Juan Lopes

It doesn't work either, because the exception occurs at MoveNext method of the enumerator. After that, the enumerator becomes unusable.

I don't think there is a solution. =/

Ayende Rahien
03/04/2010 02:29 AM by
Ayende Rahien

Michael,

Your code won't compile

David
03/04/2010 02:29 AM by
David

Stealing Michael's idea:

    public static IEnumerable

<t RobustEnumerating <t(

        IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

    {

        foreach (T item in input)

        {

            T funcedItem = default(T);

            bool worked;

            try

            {

                var funcedList = func(new [] {item});

                funcedItem = funcedList.First();

                worked = true;

            }

            catch

            {

                worked = false;

            }

            if (worked) yield return funcedItem;

        }

    }
Dennis
03/04/2010 02:29 AM by
Dennis

The problem is that the whole IEnumerable is evaluated upon the first call to the MoveNext, therefore it is not possible to have a try/catch around each.

I give up :(

vcsjones
03/04/2010 02:31 AM by
vcsjones

This is terrible, but the only way I could get this to work:

public static IEnumerable <t RobustEnumerating <t(IEnumerable <t input, Func <ienumerable<t, IEnumerable func)

{

List

<t holder = new List <t();

foreach (var i in input)

{

    try

    {

        holder.AddRange(func(Enumerable.Repeat(i, 1)));

    }

    catch

    {

    }

}

return holder;

}

Ayende Rahien
03/04/2010 02:34 AM by
Ayende Rahien

David,

That is an workable implementation, yes. It has some bugs (there isn't 1:1 mapping between result and reply)

Now try doing this without invoking the enumerator for every single item (initialization cost may be expensive).

There are at least two additional ways of doing it

pb
03/04/2010 02:34 AM by
pb

This works.. And I learned something about enumerables and exceptions today.

    private static void Main()

    {

        foreach (int i in RobustEnumerating(Enumerable.Range(0, 10), FaultyFunc))

        {

            Console.WriteLine(i);

        }

    }


    public static IEnumerable

<t RobustEnumerating <t(IEnumerable <t input, Func <ienumerable<t, IEnumerable func)

    {

        IEnumerator

<t enumerator = input.GetEnumerator();

        while (enumerator.MoveNext())

        {

            T t = enumerator.Current;


            T value = default(T);

            bool hasValue = false;

            try

            {

                value = func(new[] { t }).First();

                hasValue = true;

            }

            catch (Exception)

            {

            }


            if(hasValue)

                yield return value;

        }


    }


    public static IEnumerable

<int FaultyFunc(IEnumerable <int source)

    {

        foreach (int i in source)

        {

            yield return i / (i % 2);

        }

    }
Ayende Rahien
03/04/2010 02:34 AM by
Ayende Rahien

Dennis,

If it isn't a lazy enumerator, you can't do anything, right.

But the code I have shown IS lazy

Ayende Rahien
03/04/2010 02:35 AM by
Ayende Rahien

vsjones,

Your approach won't work with infinite enumerators, can you make it lazy?

Ayende Rahien
03/04/2010 02:36 AM by
Ayende Rahien

pb,

What happens if a function returns two output values for every input value?

Dennis
03/04/2010 02:41 AM by
Dennis

David, that wont work.

What if the FaultyFuncsaid

public static IEnumerable <int FaultyFunc(IEnumerable <int source)

{

    Console.WriteLine("Called me");

    foreach (int i in source)

    {

        yield return i/(i%2);

    }

}

You cannot presume the function is without sideeffects

Steve Py
03/04/2010 02:47 AM by
Steve Py

It ain't pretty, but it works. :)

        public static IEnumerable

<t RobustEnumerating <t(

            IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

        {

            // how to do this?

            foreach(T value in input)

            {

                T test = default(T);

                bool match = false;

                try

                {

                    test =  func(new T[] { value }).ElementAt(0);

                    match = true;

                }

                catch

                { }

                if (match)

                    yield return test;

            }

        }
Justice
03/04/2010 02:48 AM by
Justice

var e = func(input).GetEnumerator();

while(true) {

try {

    if(!e.MoveNext()) yield break;

    yield return e.Current;

}

catch { }

}

timoconnell
03/04/2010 02:49 AM by
timoconnell

Not pretty also, but...

    public static IEnumerable

<t RobustEnumerating <t(IEnumerable <t input, Func <ienumerable<t, IEnumerable func)

    {

        // how to do this?


        IEnumerator enumerator = input.GetEnumerator();

        List

<t output = new List <t();

        while (enumerator.MoveNext())

        {

            if (int.Parse(enumerator.Current.ToString()) % 2 != 0)

            {

                output.Add((T)enumerator.Current);

            }

        }


        return func((IEnumerable

<t)output);

    }
Steve Py
03/04/2010 02:50 AM by
Steve Py

dam people reply fast. :) Same as pb.

Hehe, your original challenge was to make the example work. ;P

Steve Py
03/04/2010 02:55 AM by
Steve Py

Should satisfy Ayende's adjustment:

        public static IEnumerable

<t RobustEnumerating <t(

            IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

        {

            // how to do this?

            foreach(T value in input)

            {

                List

<t results = new List <t();

                bool match = false;

                try

                {

                    results =  func(new T[] { value }).ToList();

                    match = true;

                }

                catch

                { }

                if (match)

                    foreach (T item in results)

                    {

                        yield return item;

                    }

            }

        }
Mike Thomas
03/04/2010 02:57 AM by
Mike Thomas

Should work - only calls func once so if it has side effects we are fine - works with infinite enumerations :)

public class Program {

    private static void Main() {

        foreach (int i in RobustEnumerating(Enumerable.Range(0, 10), FaultyFunc)) {

            Console.WriteLine(i);

        }

    }


    public static IEnumerable

<t RobustEnumerating <t(

        IEnumerable

<t input, Func <ienumerable<t, IEnumerable func) {

        IEnumerable

<t enumerable = func(input);

        IEnumerator

<t enumerator = null;

        try {

            enumerator = enumerable.GetEnumerator();



            while(true) {

                bool fault = false;    

                T t = default(T);


                try {

                    if(!enumerator.MoveNext()) {

                        break;

                    }


                    t = enumerator.Current;


                }

                catch {

                    var field = enumerator.GetType().GetField("<>1__state",

                                                              BindingFlags.NonPublic | BindingFlags.Instance);


                    field.SetValue(enumerator, 2);

                    fault = true;

                }


                if(!fault) {

                    yield return t;

                }

            }

        }

        finally {

            enumerator.Dispose();

        }

    }


    public static IEnumerable

<int FaultyFunc(IEnumerable <int source) {

        foreach (int i in source) {

            yield return i / (i % 2);

        }

    }

}
pb
03/04/2010 02:59 AM by
pb

Since Steve copied me I'll say ditto to his... :)

Nick Aceves
03/04/2010 03:00 AM by
Nick Aceves

@mike

You stolez my codez! ;)

@oren

This isn't possible if you think about it. If MoveNext throws you have NO idea what the state of the enumerator is. What if it's a custom Ienumerator implementation? The c# codegened ones fail gracefully, but that's not guaranteed if I write one myself.

Matt
03/04/2010 03:01 AM by
Matt

public static IEnumerable <t RobustEnumerating <t(

IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

{

if (!input.Any()) return Enumerable.Empty <t();

var result = new List <t();

try

{

  var goodResult = func(new[] {input.First()});

  result.Add(goodResult.First());

  return result.Union(RobustEnumerating(input.Skip(1), func));

}

catch

{

  return RobustEnumerating(input.Skip(1), func);

}

}

Juan Lopes
03/04/2010 03:01 AM by
Juan Lopes

Nice Mike. :)

Never thought on reseting the state of the enumerator. This sounds a bit of hack, but it works :)

yh
03/04/2010 03:04 AM by
yh

Steve Py:

you can remove the bool match flag.. as the list will be empty in case of exception...

Jay
03/04/2010 03:04 AM by
Jay

What about something like below?

It works, but I feel there has to be a cheaper way to do it.

    public static IEnumerable

<t RobustEnumerating <t(

        IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

    {

        foreach (var inputItem in input)

        {

            var tempInput = new[] { inputItem };

            var buffer = new List

<t();

            var funcEnumerator = func(tempInput).GetEnumerator();

            var flag = true;


            while (flag)

            {

                try

                {

                    if (funcEnumerator.MoveNext())

                    {

                        buffer.Add(funcEnumerator.Current);

                    }

                    else

                    {

                        flag = false;

                    }

                }

                catch { }

            }


            foreach (var tempItem in buffer)

            {

                yield return tempItem;

            }

        }

    }
Rob
03/04/2010 03:33 AM by
Rob

lazy, concise, expressive.

it'd be even better if I had an immutable list

return

            input

                .Aggregate(new List

<t(),

                    (acc, i) =>

                    {                            

                        try

                        {

                            acc.Add(func(new[] { i }).Single());

                        } catch { }

                        return acc;

                    });
Mike Thomas
03/04/2010 03:35 AM by
Mike Thomas

@Jual Lopez yeah, just a "little" hack :)

Rob
03/04/2010 03:37 AM by
Rob

Oops. Change

.Add(......Single())

to

.AddRange(.....)

to handle multiple returns for a single input

Dmitriy Nagirnyak
03/04/2010 03:40 AM by
Dmitriy Nagirnyak

public static IEnumerable <t RobustEnumerating <t(

IEnumerable

<t input,Func <ienumerable<t, IEnumerable func)

{

var res = new List

<t();

var tmp = new T[1];

foreach(var i in input) {

    try {

        tmp[0] = i;

        res.AddRange( func(tmp) );

    } catch {}

}

return res;

}

Venu
03/04/2010 04:31 AM by
Venu
 public static IEnumerable
<t RobustEnumerating
<t(
  
            IEnumerable
<t input, Func
<ienumerable<t, IEnumerable
<t> func)
  
        {
  
            // how to do this?
  
            IList
<t list = new List
<t();
  
            int counter = 0;
  
            foreach (var enumerable in input)
  
            {
  
                if (counter % 2 != 0)
  
                    list.Add(enumerable);
  
                counter++;           
  
            }
  
            input = list.AsEnumerable();
  
            return func(input);
  
  
        }
>
Steve Py
03/04/2010 04:33 AM by
Steve Py

@nb != copy. :) When I started the current post was from Ayende about Juan's not compiling. I was shocked after posting to see some 10 other responses including virtually the same solution.

@yh Yeah, could just check for the empty list... Hacking that together on my lunch break. :)

Rob
03/04/2010 04:38 AM by
Rob

Final attempt.

it turns out Aggregate isn't lazy.

The only trick here is the .ToList on the func result to force the error to occur inside the try catch rather than later when selectmany is flattening lists.

return

            input

                .SelectMany(i =>

                    {

                        try

                        {

                            return func(new[] { i }).ToList();

                        } catch {

                            return new List

<t();

                        }

                    });
Nick Aceves
03/04/2010 04:58 AM by
Nick Aceves

@mike,

Your code only works assuming the enumerator has 2 states.

Take the following, for example:

    private static void Throw() {

        throw new Exception();

    }


    public static IEnumerable

<int FaultyFunc2(IEnumerable <int source) {

        yield return 1;

        yield return 2;

        yield return 3;

        Throw();

        yield return 4;

    }

This will 5 different states, and your implementation sends it into an infinite loop.

Nick Aceves
03/04/2010 05:23 AM by
Nick Aceves

Also, Mike's code fails when the enumerator disposes of resources in a finally block. The code below sends it into an infinite loop even though there are only 2 states and state 2 is the "get me the next item" state. Attempting to continue iteration at that point leads to an ObjectDisposedException, which promptly gets swallowed and "iteration" continues.

My code-sense is telling me that unless you have an Oracle for the Halting Problem, this can't be done in such a manner that you only initialize the enumerator one time, or in a way that handles every possible enumerator implementation. You will have to impose constrains upon what "types" of enumerators you will consume. Or, better yet, remove this requirement altogether by changing the assumptions/requirements in whatever code needs this.

    public static IEnumerable

<int FaultyFunc3(IEnumerable <int source) {

        var file = new StringBuilder().AppendLine("1").AppendLine("2").AppendLine("<<hosed>>").AppendLine("3").AppendLine("4").ToString();

        var bytes = Encoding.Default.GetBytes(file);


        using(var stream = new StreamReader(new MemoryStream(bytes))) {

            while(!stream.EndOfStream)

                yield return int.Parse(stream.ReadLine());

        }

    }
Paul Batum
03/04/2010 06:02 AM by
Paul Batum

I gave up reading the suggestions, its just too annoying to read code that isn't html encoded and formatted properly. It would be better if people were posting codepaste/github gist/etc links.

Leyu Sisay
03/04/2010 06:11 AM by
Leyu Sisay

It ain't pretty.

public static IEnumerable <t RobustEnumerating <t( IEnumerable <t input, Func <ienumerable<t, IEnumerable func)

    {

        var result = new List

<t();

        var skippedInput = input;

        while(true)

        {

            var skip = 1;

            try

            {

                foreach (var value in func(skippedInput))

                {

                    skip++;

                    result.Add(value);

                }

                break;

            }

            catch

            {

                skippedInput = skippedInput.Skip(skip);

            }

        }


        return result;

    }
Bobby Diaz
03/04/2010 06:17 AM by
Bobby Diaz

How about this? (never said we couldn't add methods...)

public class Program

{

public static void Main(string[] args)

{

foreach ( int i in RobustEnumerating(Enumerable.Range(0, 10), FaultyFunc) )

{

  Console.WriteLine(i);

}


Console.WriteLine("\n");


foreach ( int i in RobustEnumerating(Enumerable.Range(0, 10).Concat(Enumerable.Range(0, 10)).OrderBy(i => i), FaultyFunc) )

{

  Console.WriteLine(i);

}


Console.Read();

}

public static IEnumerable <t RobustEnumerating <t(IEnumerable <t input, Func <ienumerable<t, IEnumerable func)

{

return input.SelectMany(i => RecursiveEnumerator(new[] { i }, func));

}

private static IEnumerable <t RecursiveEnumerator <t(IEnumerable <t input, Func <ienumerable<t, IEnumerable func)

{

var list = new List

<t();

try

{

  list.AddRange(func(input));

}

catch ( Exception )

{

  list.AddRange(RecursiveEnumerator(input.Skip(list.Count + 1), func));

}


foreach ( var item in list )

{

  yield return item;

}

}

public static IEnumerable <int FaultyFunc(IEnumerable <int source)

{

foreach ( int i in source )

{

  yield return i / ( i % 2 );

}

}

}

Andrey Titov
03/04/2010 06:38 AM by
Andrey Titov
public static IEnumerable<T> RobustEnumerating<T>(

    IEnumerable<T> input, Func<IEnumerable<T>, IEnumerable<T>> func)

{

    foreach (var item in input)

    {

        bool isValid = false;

        T current = default(T);


        try

        {

            current = func(new[] { item }).Single();

            isValid = true;

        }

        catch { }


        if (isValid)

        {

            yield return current;

        }

    }            

}
Max Uvw
03/04/2010 06:39 AM by
Max Uvw

Something something functional ;)

    public static IEnumerable

<t RobustEnumerating <t(

                IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

    {

        return input.

                Select(i => TryApply(One(i), o => func(o).First())).

                Where(m => m != Maybe

<t.Nothing).

                Select(m => m.Value);

    }


    private static IEnumerable

<t One <t(T value)

    {

        yield return value;

    }


    private static Maybe

<r TryApply <t,> (T input, Func <t,> func)

    {

        try {

            return Maybe

<r.Just(func(input));

        } catch {

            return Maybe

<r.Nothing;

        }

    }


    private sealed class Maybe

<t
{

        public static readonly Maybe

<t Nothing = new Maybe <t();

        public T Value { get; private set; }

        public static Maybe

<t Just(T value)

        {

            return new Maybe

<t {Value = value};

        }

    }
bob
03/04/2010 06:43 AM by
bob

try the following code

    public static IEnumerable

<t RobustEnumerating <t(

        IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

    {

        // how to do this?

        var newInput = from i in input

                       where Convert.ToInt32(i.ToString()) % 2 != 0

                       select i;

        return func(newInput);


    }
Demid
03/04/2010 06:49 AM by
Demid

It's a little bit ugly but it compiles, returns what you need and doing it in lazy manner:

public static IEnumerable <t Wrap <t(IEnumerator <t input)

{

    while(input.MoveNext())

        yield return input.Current;

}


public static IEnumerable

<t RobustEnumerating <t(

    IEnumerable

<t input,Func <ienumerable<t, IEnumerable func)

{

    var wrapped = Wrap(input.GetEnumerator());

    var en = func(wrapped).GetEnumerator();

    do

    {

        bool? b = MoveNext(en);

        if (b == false) yield break;

        else if (b == true) yield return en.Current;

        else en = func(wrapped).GetEnumerator();

    } while(true);

}


public static bool? MoveNext

<t(IEnumerator <t en)

{

    try

    {

        return en.MoveNext();

    } catch {}

    return null;

}
Franck
03/04/2010 06:50 AM by
Franck

What about this?

public static IEnumerable <t RobustEnumerating <t(

        IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

    {


        bool success = true;

     foreach(T el in input)

     {

         try

         {

             func(new List

<t { el }.AsEnumerable()).FirstOrDefault();

             success = true;

         }

         catch

         {

             success = false;

         }

         if (success) yield return func(new List

<t { el }.AsEnumerable()).FirstOrDefault();

     }


    }
Ajai Shankar
03/04/2010 06:50 AM by
Ajai Shankar

Since Ayende kept saying input & output need not be 1:1 here is a solution that keeps track of the input enumerator and tries to minimize initialization of faulty func.

Am sure he has some super beautiful functionally composable elegant solution up his sleeve though :-)

public static IEnumerable

<t RobustEnumerating <t(

    IEnumerable

<t input,Func <ienumerable<t, IEnumerable func)

{

    int lastIndex = -1;

    while(true) {

        var result = func(

                input.Skip(lastIndex + 1)

                .Select(item => { lastIndex += 1; return item; })

            ).GetEnumerator();


        while(true) {

            bool done = false;

            try {

                done = !result.MoveNext();

            } catch(Exception) {

                break;

            }


            if(done)

                yield break;

            else

                yield return result.Current;

        }

    }

}

Also there might be a way to keep track of the current & next in the input sequence (and avoid the Skip) but am algorithmically challenged.

Ajai

Max Uvw
03/04/2010 06:52 AM by
Max Uvw

Sorry for formatting :(

        public static IEnumerable

<t RobustEnumerating <t (

                    IEnumerable

<t input, Func <ienumerable<t , IEnumerable func)

        {

            return input.

                    Select(i => TryApply(One(i), o => func(o).First())).

                    Where(m => m != Maybe

<t .Nothing).

                    Select(m => m.Value);

        }


        private static IEnumerable

<t One <t (T value)

        {

            yield return value;

        }


        private static Maybe

<r TryApply <t,> (T input, Func <t,> func)

        {

            try {

                return Maybe

<r .Just(func(input));

            } catch {

                return Maybe

<r .Nothing;

            }

        }


        private sealed class Maybe

<t
{

            public static readonly Maybe

<t Nothing = new Maybe <t ();

            public T Value { get; private set; }

            public static Maybe

<t Just(T value)

            {

                return new Maybe

<t {Value = value};

            }

        }
Demid
03/04/2010 06:53 AM by
Demid

argggh, the site engine ate template parameters in my functions so it wont compile straight away but I'm sure it's easy to fix.

Dmitry
03/04/2010 06:54 AM by
Dmitry

Here is a version that works with multiple results:

    public static IEnumerable<T> RobustEnumerating<T>(IEnumerable<T> input, Func<IEnumerable<T>, IEnumerable<T>> func)

    {

        foreach (var item in input)

        {

            bool valid;


            IEnumerable<T>results = new T[0];


            try

            {

                results = func(new[] { item }).ToArray();

                valid = true;

            }

            catch

            {

                valid = false;

            }


            if (valid)

            {

                foreach (var result in results)

                {

                    yield return result;

                }

            }

        }

    }
Alex Yakunin
03/04/2010 07:57 AM by
Alex Yakunin

The original problem is easy to solve with interactive extensions from Rx Framework:

Instead of:

foreach (int i in RobustEnumerating(Enumerable.Range(0, 10), FaultyFunc))

use:

foreach (int i in RobustEnumerating(Enumerable.Range(0, 10), FaultyFunc).OnErrorResumeNext())

Nick Aceves
03/04/2010 07:59 AM by
Nick Aceves

I have a feeling I know why you need this...

I'm guessing that this relates to building indexes, and that you want to be able to iterate over the LINQ query that generates the index, skipping over items that (for some reason or another) cause an exception to be thrown...

If that's right, then you might try this: http://pastie.org/853146

Basically, I just reimplemented the most basic LINQ operators to swallow exceptions when enumerating. If you ditch System.Linq and just include your own implementation of those methods when you compile the index LINQ queries, you should be able to accomplish this.

Nick Aceves
03/04/2010 08:19 AM by
Nick Aceves

@Alex

I don't think that's what OnErrorResumeNext does. OnErrorResumeNext takes an enumerable of enumerables, and iterates over each one of their members in sequence. If one of enumerables throws, it moves to the next enumerable and starts iterating over its items. That doesn't allow you to iterate over a single enumerable in the way Oren described...

Ayende Rahien
03/04/2010 08:40 AM by
Ayende Rahien

Steve,

What if the func return a different number of elements?

You code with either lose some elements or fail itself when there are no elements

Ayende Rahien
03/04/2010 08:40 AM by
Ayende Rahien

Justice,

It doesn't compile

Ayende Rahien
03/04/2010 08:41 AM by
Ayende Rahien

timoconnell ,

Huh?

What happen if I give you a data source over dates?

Ayende Rahien
03/04/2010 08:44 AM by
Ayende Rahien

Mike,

Nice!

Now what happen if I pass in an enumerable that isn't using yield?

Or a different enumerable where state might be something else?

Ayende Rahien
03/04/2010 08:45 AM by
Ayende Rahien

Nick,

I have three different implementations that work :-)

Ayende Rahien
03/04/2010 08:46 AM by
Ayende Rahien

Mike,

What if you can only traverse the enumerable once?

Ayende Rahien
03/04/2010 08:48 AM by
Ayende Rahien

Jay,

This doesn't work with the function provided.

Those enumerables are stateful, and will stop on the first exception

Ayende Rahien
03/04/2010 08:49 AM by
Ayende Rahien

Venu,

The function cannot have any knowledge of the methods I pass to it.

Ayende Rahien
03/04/2010 08:52 AM by
Ayende Rahien

Nick,

Very good observation!

You can still do a lot with it, though.

See my next few posts

Ayende Rahien
03/04/2010 08:53 AM by
Ayende Rahien

Bobby,

You iterate over the source enumerable more than once

Ayende Rahien
03/04/2010 08:54 AM by
Ayende Rahien

Demid,

You hit my second implementation right on the money, but did it much more elegantly!

Nice work

Ayende Rahien
03/04/2010 08:56 AM by
Ayende Rahien

Ajai,

You are iterating over the input enum more than once

Ayende Rahien
03/04/2010 09:01 AM by
Ayende Rahien

Nick,

You are correct about how I run into this.

Wait for the next post on the subject, though, I'll give it full coverage.

And it looks like Linq actually make this easier

Nick Aceves
03/04/2010 09:15 AM by
Nick Aceves

Oren,

I bet for each one of your implementations, I can devise an enumerator implementation for which they fail ;)

I really don't think this can work in a perfectly general sense (that is, for any enumerator implementation); assumptions have to be made as to the behavior of the enumerator under certain conditions.

But, if you're expecting to get compiler-generated enumerators, or enumerators from LINQ chains, then you can, indeed, make assumptions about how they work.

Kenneth
03/04/2010 09:56 AM by
Kenneth
    public static IEnumerable

<t RobustEnumerating <t(

        IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

    {

        var dd = input.Where(x =>

        {

            string s = x.ToString();

            int i = 0;

            if (int.TryParse(s, out i))

            {

                if (i % 2 == 0) return false;

                return true;

            }

            return false;

        });

        // how to do this?

        return func(dd);

    }
Koen
03/04/2010 09:59 AM by
Koen

Pretty sure this is not what you want but it works:

return func(input.Cast <int().Where(i => i % 2 == 1).Cast <t());

Ayende Rahien
03/04/2010 09:59 AM by
Ayende Rahien

Kenneth,

You can't assume the same enumerating function.

Benny Thomas
03/04/2010 10:03 AM by
Benny Thomas

Ugly code that needs some tuning.

public static IEnumerable <t RobustEnumerating <t(

    IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

{


    foreach(T a in input)

    {

        List

<t results = new List <t();

        try

        {

            results.AddRange(func(new List

<t {a}));

        }

        catch

        {

            // Do some faulthandling here

        }

        foreach (var result in results)

        {

            yield return result;

        }

    }        

}
Alex Yakunin
03/04/2010 10:14 AM by
Alex Yakunin

@ Nick Aceves

You're right - I really made a mistake. Btw, such OnErrorResumeNext is fitting very well on their sequence concept (throw terminates a sequence).

So now it looks like a tricky case to implement this fully on default LINQ combinators and Rx.

Koen
03/04/2010 10:28 AM by
Koen

My generic type parameters weren't automatically HTML encoded so here's a second attempt encoding them myself:

return func(input.Cast().Where(i => i % 2 == 1).Cast());

Sergey Kachkovsky
03/04/2010 11:14 AM by
Sergey Kachkovsky

public static IEnumerable <t RobustEnumerating <t(

        IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

    {

        IEnumerator

<t enumerator = func(input).GetEnumerator();

        do

        {

            bool valid = true;

            try

            {

                if (!enumerator.MoveNext()) yield break;

            }

            catch

            {

                valid = false;

            }

            if (valid)

                yield return enumerator.Current;

        } while (true);

    }
Mike Doherty
03/04/2010 11:26 AM by
Mike Doherty

Ok. Here is my solution:

public static IEnumerable

<t RobustEnumerating <t (

    IEnumerable

<t input, Func <ienumerable<t , IEnumerable func)

{

    var rowsToSkipOnError = 1;

    var enumerationToWrap = func(input).GetEnumerator();

    while (true)

    {

        T nextValue;

        try

        {

            if (!enumerationToWrap.MoveNext())

                yield break;


            nextValue = enumerationToWrap.Current;

        }

        catch (Exception)

        {

            // This input failed so skip to the next one

            input = input.Skip(rowsToSkipOnError);

            enumerationToWrap = func(input).GetEnumerator();

            rowsToSkipOnError = 1;

            continue;

        }


        rowsToSkipOnError++;

        yield return nextValue;

    }

}
Arielr
03/04/2010 11:30 AM by
Arielr

return input.SelectMany(x =>

                        {

                            try

                            {

                                return func(x.AsEnumerableX()).ToArray();

                            }

                            catch

                            {

                                return Enumerable.Empty

<t();

                            }

                        });
Mike Doherty
03/04/2010 11:49 AM by
Mike Doherty

This time I've escaped the angle brackets to see if that slips past the comment sanitizer.

Looking over Dimid's solution though I realize that I am repeatedly enumerating over the input which could have a performance impact if getting these values was expensive. I love the way that he wraps the input to preserve it's state.

public static IEnumerable<T> RobustEnumerating<T>(

   IEnumerable<T> input, Func<IEnumerable<T>, IEnumerable<T>> func)

{

   var rowsToSkipOnError = 1;

   var enumerationToWrap = func(input).GetEnumerator();

   while (true)

   {

       T nextValue;

       try

       {

           if (!enumerationToWrap.MoveNext())

               yield break;


           nextValue = enumerationToWrap.Current;

       }

       catch (Exception)

       {

           // This input failed so skip to the next one

           input = input.Skip(rowsToSkipOnError);

           enumerationToWrap = func(input).GetEnumerator();

           rowsToSkipOnError = 1;

           continue;

       }


       rowsToSkipOnError++;

       yield return nextValue;

   }

}
Frank
03/04/2010 12:17 PM by
Frank

Without having looked at previous posted entries, this should do the trick, while still supporting infinite enumerations:

public static IEnumerable <t RobustEnumerating <t
(

IEnumerable

<t input,

Func

<ienumerable<t, IEnumerable func

)

{

var singleInputElementArray = new T[1];

foreach (var inputElement in input)

{

    singleInputElementArray[0] = inputElement;


    IEnumerator

<t enumerator = func(singleInputElementArray).GetEnumerator();

    bool success;

    do

    {

        success = false;


        try

        {

            if (enumerator.MoveNext())

                success = true;

        }

        catch

        {

        }


        if (success)

            yield return enumerator.Current;

    }

    while (success);

}

}

Sergey Shishkin
03/04/2010 01:00 PM by
Sergey Shishkin
        return input.SelectMany(x =>

        {

            try

            {

                return new[]{func(new[] { x }).FirstOrDefault()};

            }

            catch

            {

                return new T[0];

            }

        });
tobi
03/04/2010 01:16 PM by
tobi

I cannot believe that 95% of all answers are just wrong. It is not that hard... If your call to movenext is not in a try-catch, your code is wrong. if you use foreach, your code is wrong because an exception will skip the rest of the enumeration. Do not call func multiple times, it is not necessary. It will enumerate the sequence multiple times.

Nice challenge however.

Mike Thomas
03/04/2010 03:10 PM by
Mike Thomas

@ayende or if MoveNext just sets some other flag before it throws or if MoveNext disposes of itself before it throws or if MoveNext does something like this (assuming for the moment that I still reset the state correctly):

yield return memberCounterThatINeedToBeUniquePerIncomingItem + transformThatWontThrow();

yield return memberCounterThatINeedToBeUniquePerIncomingItem + transformThatWillThrow();

memberCounterThatINeedToBeUniquePerIncomingItem++;

Basically, like Nick says, if we have the requirement that the func can be called only once (which makes sense since it could be keeping track of its position or could make a million expensive remote calls before returning the first item) you can't solve this in a generic way.

tobi
03/04/2010 03:33 PM by
tobi

the func transforms a non-throwing sequence into a throwing sequence. you can solve it calling it only once.

Michal
03/04/2010 03:38 PM by
Michal

return input.Select(i =>

                    {

                        try

                        {

                            return func(new[] { i }).First();

                        }

                        catch

                        {

                            return default(T);

                        }

                    }).

                Where(i => !i.Equals(default(T)));
Ajai Shankar
03/04/2010 05:09 PM by
Ajai Shankar

@Ayende "Ajai, You are iterating over the input enum more than once"

Here is what I think works (I had to add a method, don't know if it is allowed)

Very nice problem, do you expect the input enumerable to throw too?

Now I need to go get some real work done :-)

Looking forward to what you have, can't take the suspense any longer!

static IEnumerable

<t EnumOnce <t(IEnumerator <t e)

{

    while(e.MoveNext())

        yield return e.Current;

}


public static IEnumerable

<t RobustEnumerating <t(

    IEnumerable

<t input,Func <ienumerable<t, IEnumerable func)

{

    input = EnumOnce(input.GetEnumerator());


    var result = func(input).GetEnumerator();

    while(true) {

        try {

            if(!result.MoveNext())

                break;

        } catch(Exception) {

            result = func(input).GetEnumerator();

            continue;

        }

        yield return result.Current;

    }

}
Kvasov Roman
03/04/2010 05:44 PM by
Kvasov Roman

I think its more robust implementation with additional class, but it call function only once so if it contains some initialization logic it doesnt break.

using System;

using System.Collections;

using System.Collections.Generic;

using System.Linq;


public class Program

{

    public class StatefullEnumerable

<t : IEnumerable <t {

        private IEnumerator

<t enumerator;

        public StatefullEnumerable(IEnumerable

<t enumerable) {

            enumerator = enumerable.GetEnumerator();

        }


        public IEnumerator

<t GetEnumerator() {

            return enumerator;

        }


        IEnumerator IEnumerable.GetEnumerator() {

            return GetEnumerator();

        }

    }


    private static void Main()

    {

        foreach (int i in RobustEnumerating(Enumerable.Range(0, 10), FaultyFunc))

        {

            Console.WriteLine(i);

        }

    }


    public static IEnumerable

<t RobustEnumerating <t (

        IEnumerable

<t input,Func <ienumerable<t , IEnumerable func)

    {

        var enumerable = func(new StatefullEnumerable

<t (input));

        var enumerator = enumerable.GetEnumerator();

        while (true) {

            var fail = false;

            var value = default(T);

            try {

                if (!enumerator.MoveNext())

                    yield break;

                value = enumerator.Current;

            }

            catch (Exception) {

                fail = true;

                enumerator = enumerable.GetEnumerator();

            }

            if (!fail)

                yield return value;

        }

        // how to do this?

    }


    public static IEnumerable

<int FaultyFunc(IEnumerable <int source)

    {

        foreach (int i in source)

        {

            yield return i/(i%2);

        }

    }

}
tobi
03/04/2010 06:52 PM by
tobi

Wow I will remember this challenge and use it as an interview question. It seems like everyone gets it wrong the first time so you can discuss the problem with the candidate to see if they are smart or not.

Also I do not understand why everyone seems to restart the sequence when an error has occurred. Just do nothing or else the sequence will restart over and over.

I do not understand the purpose of StatefullEnumerable. It seems to do nothing.

James C-S
03/04/2010 11:46 PM by
James C-S

This is probably not in the spirit of the challenge, but ...

public static IEnumerable

<t RobustEnumerating <t(

    IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

{

    return func(input.Cast

<int().Where(x => x % 2 != 0).Cast <t());

}

...works just fine. Of course it does break encapsulation. :-)

James C-S
03/05/2010 12:10 AM by
James C-S

OK. I felt bad about me previous answer that broke encapsulation. Here's an answer that doesn't.

    public static IEnumerable

<t RobustEnumerating <t(

        IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

    {

        Func

<ienumerable<t, IEnumerable safeFunc = source =>

        {

            var @return = (IEnumerable

<t)null;

            try { @return = func(source).ToArray(); }

            catch (System.DivideByZeroException) { }

            return @return;

        };

        return (from i2 in

                    (from i in input

                     select new T[] { i })

                let o2 = safeFunc(i2)

                where o2 != null

                select o2).SelectMany(o => o);

    }
James C-S
03/05/2010 12:11 AM by
James C-S

Whoops! Now I feel bad about me, err my, grammar... ;-)

Demid
03/05/2010 12:13 AM by
Demid

I can't believe so many people are so lazy to compile their code at least.

It's not a crime do not know that try/catch block isn't compatible with yield statement but you can at least try to compile it.

Download SnippetCompiler. It is outdated project but it really really good to test your ideas whenever you're not sure how compiler or .net behaves in a particular situation because. It loaded instantly and I ran Oren's example in two seconds after I saw it.

Dmitriy Nagirnyak
03/05/2010 12:57 AM by
Dmitriy Nagirnyak

@Demind, I use LINQPad for that purpose. Copy-paste code and it just runs.

Ajai Shankar
03/05/2010 02:01 AM by
Ajai Shankar

@Demid & Dmitriy: vi & csc works too :-)

Ken Tong
03/05/2010 02:38 AM by
Ken Tong

How about this?

public static IEnumerable<T> RobustEnumerating<T>(

    IEnumerable<T> input, Func<IEnumerable<T>, IEnumerable<T>> func)

{

    IEnumerable<T> results = new T[0];


    foreach (T item in input)

    {

        try

        {

            foreach (T value in func(new[] { item }))

                results = results.Concat(new[] { value });

        }

        catch { }

    }


    foreach (T result in results)

        yield return result;

}
Allan PArker
03/05/2010 11:04 AM by
Allan PArker

Hi,

Here is my attempt at this:

    public static IEnumerable

<t RobustEnumerating <t(IEnumerable <t input, Func <ienumerable<t, IEnumerable func)

    {

        foreach (T item in input)

        {

            IEnumerable

<t result;

            var list = new List

<t { item };

            try

            {

                result = func(list);

                var test = result.First();

            }

            catch

            {

                continue;

            }


            yield return result.First();

        }

    }
Thomas Eyde
03/05/2010 01:59 PM by
Thomas Eyde

It looks like the solution is a variation over iterating the input, because that's the only thing we can control, evaluate each item and only return the results of those evaluations that don't fail.

Here's my attempt:

            foreach (var item in input)

            {

                T[] evaluatedItems;

                try

                {

                    var itemAsEnumerable = new[] { item };

                    evaluatedItems = func(itemAsEnumerable).ToArray();

                }

                catch (Exception)

                {

                    continue;

                }


                foreach (var evaluated in evaluatedItems)

                {

                    yield return evaluated;

                }

            }
Chris Martin
03/05/2010 06:30 PM by
Chris Martin

public static IEnumerable <int RobustEnumerating(IEnumerable <int input, Func <ienumerable<int, IEnumerable func)

{

return func(input.ToList().ConvertAll(t => int.Parse(t.ToString())).Where(i => i%2 > 0));

}

lol

firefly
03/05/2010 09:44 PM by
firefly

@ tobi, it's not hard but I wouldn't say it's intuitive either. Especially for an interview question unless you've wrote library code where you use MoveNext a lot and have an intimate understanding of how it really work.

Now if you've been exposed to this before somewhere else then it's extremely easy :). Thanks Oren and everyone else that comments. I learned a lot from the exposure... especially from Demid code, very elegant indeed.

sergio
03/09/2010 08:01 AM by
sergio

Maybe its not the best, but it works

public class Program

{

    private static void Main()

    {

        foreach (int i in RobustEnumerating(Enumerable.Range(0, 10), FaultyFunc))

        {

            Console.WriteLine(i);

        }

        Console.ReadLine();

    }


    public static IEnumerable

<t RobustEnumerating <t(

        IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

    {

        // how to do this?

        var enumerator = func(input).GetEnumerator();

        while (SafeMoveNextEnumerator(input, func, ref enumerator))

        {

            yield return enumerator.Current;

        }




    }

    static int skip = 0;


    public static bool SafeMoveNextEnumerator

<t(IEnumerable <t input, Func <ienumerable<t, IEnumerable func, ref IEnumerator <t safe)

    {

        while (true)

        {

            skip++;

            try

            {

                return safe.MoveNext();

            }

            catch

            {

                var e = Enumerable.Skip(input, skip);

                safe = func(e).GetEnumerator();

            }

        }

    }


    public static IEnumerable

<int FaultyFunc(IEnumerable <int source)

    {

        foreach (int i in source)

        {

            yield return i / (i % 2);

        }

    }

}
James
03/11/2010 10:07 AM by
James

Simplest possible solution ...

    public static IEnumerable

<int RobustEnumerating <t(IEnumerable <t input, Func <ienumerable<t, IEnumerable func)

    {

        return new[] { 1, 3, 5, 7, 9 };

    }
Comments have been closed on this topic.