Ayende @ Rahien

Hi!
My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:

ayende@ayende.com

+972 52-548-6969

, @ Q c

Posts: 08 | Comments: 17

filter by tags archive

ChallengeRobust enumeration over external code

time to read 2 min | 378 words

Here is an interesting little problem:

public class Program
{
    private static void Main()
    {
        foreach (int i in RobustEnumerating(Enumerable.Range(0, 10), FaultyFunc))
        {
            Console.WriteLine(i);
        }
    }

    public static IEnumerable<T> RobustEnumerating<T>(
        IEnumerable<T> input,Func<IEnumerable<T>, IEnumerable<T>> func)
    {
        // how to do this?
        return func(input);

    }

    public static IEnumerable<int> FaultyFunc(IEnumerable<int> source)
    {
        foreach (int i in source)
        {
            yield return i/(i%2);
        }
    }
}

This code should not throw, but print:

1
3
5
7
9

Can you make this happen? You can only change the RobustEnumerating method, nothing else in the code

More posts in "Challenge" series:

  1. (28 Apr 2015) What is the meaning of this change?
  2. (26 Sep 2013) Spot the bug
  3. (27 May 2013) The problem of locking down tasks…
  4. (17 Oct 2011) Minimum number of round trips
  5. (23 Aug 2011) Recent Comments with Future Posts
  6. (02 Aug 2011) Modifying execution approaches
  7. (29 Apr 2011) Stop the leaks
  8. (23 Dec 2010) This code should never hit production
  9. (17 Dec 2010) Your own ThreadLocal
  10. (03 Dec 2010) Querying relative information with RavenDB
  11. (29 Jun 2010) Find the bug
  12. (23 Jun 2010) Dynamically dynamic
  13. (28 Apr 2010) What killed the application?
  14. (19 Mar 2010) What does this code do?
  15. (04 Mar 2010) Robust enumeration over external code
  16. (16 Feb 2010) Premature optimization, and all of that…
  17. (12 Feb 2010) Efficient querying
  18. (10 Feb 2010) Find the resource leak
  19. (21 Oct 2009) Can you spot the bug?
  20. (18 Oct 2009) Why is this wrong?
  21. (17 Oct 2009) Write the check in comment
  22. (15 Sep 2009) NH Prof Exporting Reports
  23. (02 Sep 2009) The lazy loaded inheritance many to one association OR/M conundrum
  24. (01 Sep 2009) Why isn’t select broken?
  25. (06 Aug 2009) Find the bug fixes
  26. (26 May 2009) Find the bug
  27. (14 May 2009) multi threaded test failure
  28. (11 May 2009) The regex that doesn’t match
  29. (24 Mar 2009) probability based selection
  30. (13 Mar 2009) C# Rewriting
  31. (18 Feb 2009) write a self extracting program
  32. (04 Sep 2008) Don't stop with the first DSL abstraction
  33. (02 Aug 2008) What is the problem?
  34. (28 Jul 2008) What does this code do?
  35. (26 Jul 2008) Find the bug fix
  36. (05 Jul 2008) Find the deadlock
  37. (03 Jul 2008) Find the bug
  38. (02 Jul 2008) What is wrong with this code
  39. (05 Jun 2008) why did the tests fail?
  40. (27 May 2008) Striving for better syntax
  41. (13 Apr 2008) calling generics without the generic type
  42. (12 Apr 2008) The directory tree
  43. (24 Mar 2008) Find the version
  44. (21 Jan 2008) Strongly typing weakly typed code
  45. (28 Jun 2007) Windsor Null Object Dependency Facility

Comments

Juan Lopes

public static IEnumerable <t RobustEnumerating <t(

    IEnumerable

<t input,Func <ienumerable<t, IEnumerable func)

{

    foreach(T t in func(input)) {

        try { yield return t; }

        catch { }

    }

}
Ayende Rahien

Juan,

That code doesn't compiles

Michael Stum

Not tested and ugly and possibly cheating, but:

public static IEnumerable <t RobustEnumerating <t(

    IEnumerable

<t input,Func <ienumerable<t, IEnumerable func)

{


  foreach(T obj in input) {

      try {

          var newList = new List

<t();

          newList.Add(obj);

          yield return func(newList)

      } catch {}

      }

}
Juan Lopes

It doesn't work either, because the exception occurs at MoveNext method of the enumerator. After that, the enumerator becomes unusable.

I don't think there is a solution. =/

Ayende Rahien

Michael,

Your code won't compile

David

Stealing Michael's idea:

    public static IEnumerable

<t RobustEnumerating <t(

        IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

    {

        foreach (T item in input)

        {

            T funcedItem = default(T);

            bool worked;

            try

            {

                var funcedList = func(new [] {item});

                funcedItem = funcedList.First();

                worked = true;

            }

            catch

            {

                worked = false;

            }

            if (worked) yield return funcedItem;

        }

    }
Dennis

The problem is that the whole IEnumerable is evaluated upon the first call to the MoveNext, therefore it is not possible to have a try/catch around each.

I give up :(

vcsjones

This is terrible, but the only way I could get this to work:

public static IEnumerable <t RobustEnumerating <t(IEnumerable <t input, Func <ienumerable<t, IEnumerable func)

{

List

<t holder = new List <t();

foreach (var i in input)

{

    try

    {

        holder.AddRange(func(Enumerable.Repeat(i, 1)));

    }

    catch

    {

    }

}

return holder;

}

Ayende Rahien

David,

That is an workable implementation, yes. It has some bugs (there isn't 1:1 mapping between result and reply)

Now try doing this without invoking the enumerator for every single item (initialization cost may be expensive).

There are at least two additional ways of doing it

pb
pb

This works.. And I learned something about enumerables and exceptions today.

    private static void Main()

    {

        foreach (int i in RobustEnumerating(Enumerable.Range(0, 10), FaultyFunc))

        {

            Console.WriteLine(i);

        }

    }


    public static IEnumerable

<t RobustEnumerating <t(IEnumerable <t input, Func <ienumerable<t, IEnumerable func)

    {

        IEnumerator

<t enumerator = input.GetEnumerator();

        while (enumerator.MoveNext())

        {

            T t = enumerator.Current;


            T value = default(T);

            bool hasValue = false;

            try

            {

                value = func(new[] { t }).First();

                hasValue = true;

            }

            catch (Exception)

            {

            }


            if(hasValue)

                yield return value;

        }


    }


    public static IEnumerable

<int FaultyFunc(IEnumerable <int source)

    {

        foreach (int i in source)

        {

            yield return i / (i % 2);

        }

    }
Ayende Rahien

Dennis,

If it isn't a lazy enumerator, you can't do anything, right.

But the code I have shown IS lazy

Ayende Rahien

vsjones,

Your approach won't work with infinite enumerators, can you make it lazy?

Ayende Rahien

pb,

What happens if a function returns two output values for every input value?

Dennis

David, that wont work.

What if the FaultyFuncsaid

public static IEnumerable <int FaultyFunc(IEnumerable <int source)

{

    Console.WriteLine("Called me");

    foreach (int i in source)

    {

        yield return i/(i%2);

    }

}

You cannot presume the function is without sideeffects

Steve Py

It ain't pretty, but it works. :)

        public static IEnumerable

<t RobustEnumerating <t(

            IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

        {

            // how to do this?

            foreach(T value in input)

            {

                T test = default(T);

                bool match = false;

                try

                {

                    test =  func(new T[] { value }).ElementAt(0);

                    match = true;

                }

                catch

                { }

                if (match)

                    yield return test;

            }

        }
Justice

var e = func(input).GetEnumerator();

while(true) {

try {

    if(!e.MoveNext()) yield break;

    yield return e.Current;

}

catch { }

}

timoconnell

Not pretty also, but...

    public static IEnumerable

<t RobustEnumerating <t(IEnumerable <t input, Func <ienumerable<t, IEnumerable func)

    {

        // how to do this?


        IEnumerator enumerator = input.GetEnumerator();

        List

<t output = new List <t();

        while (enumerator.MoveNext())

        {

            if (int.Parse(enumerator.Current.ToString()) % 2 != 0)

            {

                output.Add((T)enumerator.Current);

            }

        }


        return func((IEnumerable

<t)output);

    }
Steve Py

dam people reply fast. :) Same as pb.

Hehe, your original challenge was to make the example work. ;P

Steve Py

Should satisfy Ayende's adjustment:

        public static IEnumerable

<t RobustEnumerating <t(

            IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

        {

            // how to do this?

            foreach(T value in input)

            {

                List

<t results = new List <t();

                bool match = false;

                try

                {

                    results =  func(new T[] { value }).ToList();

                    match = true;

                }

                catch

                { }

                if (match)

                    foreach (T item in results)

                    {

                        yield return item;

                    }

            }

        }
Mike Thomas

Should work - only calls func once so if it has side effects we are fine - works with infinite enumerations :)

public class Program {

    private static void Main() {

        foreach (int i in RobustEnumerating(Enumerable.Range(0, 10), FaultyFunc)) {

            Console.WriteLine(i);

        }

    }


    public static IEnumerable

<t RobustEnumerating <t(

        IEnumerable

<t input, Func <ienumerable<t, IEnumerable func) {

        IEnumerable

<t enumerable = func(input);

        IEnumerator

<t enumerator = null;

        try {

            enumerator = enumerable.GetEnumerator();



            while(true) {

                bool fault = false;    

                T t = default(T);


                try {

                    if(!enumerator.MoveNext()) {

                        break;

                    }


                    t = enumerator.Current;


                }

                catch {

                    var field = enumerator.GetType().GetField("<>1__state",

                                                              BindingFlags.NonPublic | BindingFlags.Instance);


                    field.SetValue(enumerator, 2);

                    fault = true;

                }


                if(!fault) {

                    yield return t;

                }

            }

        }

        finally {

            enumerator.Dispose();

        }

    }


    public static IEnumerable

<int FaultyFunc(IEnumerable <int source) {

        foreach (int i in source) {

            yield return i / (i % 2);

        }

    }

}
pb
pb

Since Steve copied me I'll say ditto to his... :)

Nick Aceves

@mike

You stolez my codez! ;)

@oren

This isn't possible if you think about it. If MoveNext throws you have NO idea what the state of the enumerator is. What if it's a custom Ienumerator implementation? The c# codegened ones fail gracefully, but that's not guaranteed if I write one myself.

Matt

public static IEnumerable <t RobustEnumerating <t(

IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

{

if (!input.Any()) return Enumerable.Empty <t();

var result = new List <t();

try

{

  var goodResult = func(new[] {input.First()});

  result.Add(goodResult.First());

  return result.Union(RobustEnumerating(input.Skip(1), func));

}

catch

{

  return RobustEnumerating(input.Skip(1), func);

}

}

Juan Lopes

Nice Mike. :)

Never thought on reseting the state of the enumerator. This sounds a bit of hack, but it works :)

yh
yh

Steve Py:

you can remove the bool match flag.. as the list will be empty in case of exception...

Jay
Jay

What about something like below?

It works, but I feel there has to be a cheaper way to do it.

    public static IEnumerable

<t RobustEnumerating <t(

        IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

    {

        foreach (var inputItem in input)

        {

            var tempInput = new[] { inputItem };

            var buffer = new List

<t();

            var funcEnumerator = func(tempInput).GetEnumerator();

            var flag = true;


            while (flag)

            {

                try

                {

                    if (funcEnumerator.MoveNext())

                    {

                        buffer.Add(funcEnumerator.Current);

                    }

                    else

                    {

                        flag = false;

                    }

                }

                catch { }

            }


            foreach (var tempItem in buffer)

            {

                yield return tempItem;

            }

        }

    }
Rob
Rob

lazy, concise, expressive.

it'd be even better if I had an immutable list

return

            input

                .Aggregate(new List

<t(),

                    (acc, i) =>

                    {                            

                        try

                        {

                            acc.Add(func(new[] { i }).Single());

                        } catch { }

                        return acc;

                    });
Mike Thomas

@Jual Lopez yeah, just a "little" hack :)

Rob
Rob

Oops. Change

.Add(......Single())

to

.AddRange(.....)

to handle multiple returns for a single input

Dmitriy Nagirnyak

public static IEnumerable <t RobustEnumerating <t(

IEnumerable

<t input,Func <ienumerable<t, IEnumerable func)

{

var res = new List

<t();

var tmp = new T[1];

foreach(var i in input) {

    try {

        tmp[0] = i;

        res.AddRange( func(tmp) );

    } catch {}

}

return res;

}

Venu
 public static IEnumerable
<t RobustEnumerating
<t(
  
            IEnumerable
<t input, Func
<ienumerable<t, IEnumerable
<t> func)
  
        {
  
            // how to do this?
  
            IList
<t list = new List
<t();
  
            int counter = 0;
  
            foreach (var enumerable in input)
  
            {
  
                if (counter % 2 != 0)
  
                    list.Add(enumerable);
  
                counter++;           
  
            }
  
            input = list.AsEnumerable();
  
            return func(input);
  
  
        }
>
Steve Py

@nb != copy. :) When I started the current post was from Ayende about Juan's not compiling. I was shocked after posting to see some 10 other responses including virtually the same solution.

@yh Yeah, could just check for the empty list... Hacking that together on my lunch break. :)

Rob
Rob

Final attempt.

it turns out Aggregate isn't lazy.

The only trick here is the .ToList on the func result to force the error to occur inside the try catch rather than later when selectmany is flattening lists.

return

            input

                .SelectMany(i =>

                    {

                        try

                        {

                            return func(new[] { i }).ToList();

                        } catch {

                            return new List

<t();

                        }

                    });
Nick Aceves

@mike,

Your code only works assuming the enumerator has 2 states.

Take the following, for example:

    private static void Throw() {

        throw new Exception();

    }


    public static IEnumerable

<int FaultyFunc2(IEnumerable <int source) {

        yield return 1;

        yield return 2;

        yield return 3;

        Throw();

        yield return 4;

    }

This will 5 different states, and your implementation sends it into an infinite loop.

Nick Aceves

Also, Mike's code fails when the enumerator disposes of resources in a finally block. The code below sends it into an infinite loop even though there are only 2 states and state 2 is the "get me the next item" state. Attempting to continue iteration at that point leads to an ObjectDisposedException, which promptly gets swallowed and "iteration" continues.

My code-sense is telling me that unless you have an Oracle for the Halting Problem, this can't be done in such a manner that you only initialize the enumerator one time, or in a way that handles every possible enumerator implementation. You will have to impose constrains upon what "types" of enumerators you will consume. Or, better yet, remove this requirement altogether by changing the assumptions/requirements in whatever code needs this.

    public static IEnumerable

<int FaultyFunc3(IEnumerable <int source) {

        var file = new StringBuilder().AppendLine("1").AppendLine("2").AppendLine("<<hosed>>").AppendLine("3").AppendLine("4").ToString();

        var bytes = Encoding.Default.GetBytes(file);


        using(var stream = new StreamReader(new MemoryStream(bytes))) {

            while(!stream.EndOfStream)

                yield return int.Parse(stream.ReadLine());

        }

    }
Paul Batum

I gave up reading the suggestions, its just too annoying to read code that isn't html encoded and formatted properly. It would be better if people were posting codepaste/github gist/etc links.

Leyu Sisay

It ain't pretty.

public static IEnumerable <t RobustEnumerating <t( IEnumerable <t input, Func <ienumerable<t, IEnumerable func)

    {

        var result = new List

<t();

        var skippedInput = input;

        while(true)

        {

            var skip = 1;

            try

            {

                foreach (var value in func(skippedInput))

                {

                    skip++;

                    result.Add(value);

                }

                break;

            }

            catch

            {

                skippedInput = skippedInput.Skip(skip);

            }

        }


        return result;

    }
Bobby Diaz

How about this? (never said we couldn't add methods...)

public class Program

{

public static void Main(string[] args)

{

foreach ( int i in RobustEnumerating(Enumerable.Range(0, 10), FaultyFunc) )

{

  Console.WriteLine(i);

}


Console.WriteLine("\n");


foreach ( int i in RobustEnumerating(Enumerable.Range(0, 10).Concat(Enumerable.Range(0, 10)).OrderBy(i => i), FaultyFunc) )

{

  Console.WriteLine(i);

}


Console.Read();

}

public static IEnumerable <t RobustEnumerating <t(IEnumerable <t input, Func <ienumerable<t, IEnumerable func)

{

return input.SelectMany(i => RecursiveEnumerator(new[] { i }, func));

}

private static IEnumerable <t RecursiveEnumerator <t(IEnumerable <t input, Func <ienumerable<t, IEnumerable func)

{

var list = new List

<t();

try

{

  list.AddRange(func(input));

}

catch ( Exception )

{

  list.AddRange(RecursiveEnumerator(input.Skip(list.Count + 1), func));

}


foreach ( var item in list )

{

  yield return item;

}

}

public static IEnumerable <int FaultyFunc(IEnumerable <int source)

{

foreach ( int i in source )

{

  yield return i / ( i % 2 );

}

}

}

Andrey Titov
public static IEnumerable<T> RobustEnumerating<T>(

    IEnumerable<T> input, Func<IEnumerable<T>, IEnumerable<T>> func)

{

    foreach (var item in input)

    {

        bool isValid = false;

        T current = default(T);


        try

        {

            current = func(new[] { item }).Single();

            isValid = true;

        }

        catch { }


        if (isValid)

        {

            yield return current;

        }

    }            

}
Max Uvw

Something something functional ;)

    public static IEnumerable

<t RobustEnumerating <t(

                IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

    {

        return input.

                Select(i => TryApply(One(i), o => func(o).First())).

                Where(m => m != Maybe

<t.Nothing).

                Select(m => m.Value);

    }


    private static IEnumerable

<t One <t(T value)

    {

        yield return value;

    }


    private static Maybe

<r TryApply <t,> (T input, Func <t,> func)

    {

        try {

            return Maybe

<r.Just(func(input));

        } catch {

            return Maybe

<r.Nothing;

        }

    }


    private sealed class Maybe

<t
{

        public static readonly Maybe

<t Nothing = new Maybe <t();

        public T Value { get; private set; }

        public static Maybe

<t Just(T value)

        {

            return new Maybe

<t {Value = value};

        }

    }
bob
bob

try the following code

    public static IEnumerable

<t RobustEnumerating <t(

        IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

    {

        // how to do this?

        var newInput = from i in input

                       where Convert.ToInt32(i.ToString()) % 2 != 0

                       select i;

        return func(newInput);


    }
Demid

It's a little bit ugly but it compiles, returns what you need and doing it in lazy manner:

public static IEnumerable <t Wrap <t(IEnumerator <t input)

{

    while(input.MoveNext())

        yield return input.Current;

}


public static IEnumerable

<t RobustEnumerating <t(

    IEnumerable

<t input,Func <ienumerable<t, IEnumerable func)

{

    var wrapped = Wrap(input.GetEnumerator());

    var en = func(wrapped).GetEnumerator();

    do

    {

        bool? b = MoveNext(en);

        if (b == false) yield break;

        else if (b == true) yield return en.Current;

        else en = func(wrapped).GetEnumerator();

    } while(true);

}


public static bool? MoveNext

<t(IEnumerator <t en)

{

    try

    {

        return en.MoveNext();

    } catch {}

    return null;

}
Franck

What about this?

public static IEnumerable <t RobustEnumerating <t(

        IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

    {


        bool success = true;

     foreach(T el in input)

     {

         try

         {

             func(new List

<t { el }.AsEnumerable()).FirstOrDefault();

             success = true;

         }

         catch

         {

             success = false;

         }

         if (success) yield return func(new List

<t { el }.AsEnumerable()).FirstOrDefault();

     }


    }
Ajai Shankar

Since Ayende kept saying input & output need not be 1:1 here is a solution that keeps track of the input enumerator and tries to minimize initialization of faulty func.

Am sure he has some super beautiful functionally composable elegant solution up his sleeve though :-)

public static IEnumerable

<t RobustEnumerating <t(

    IEnumerable

<t input,Func <ienumerable<t, IEnumerable func)

{

    int lastIndex = -1;

    while(true) {

        var result = func(

                input.Skip(lastIndex + 1)

                .Select(item => { lastIndex += 1; return item; })

            ).GetEnumerator();


        while(true) {

            bool done = false;

            try {

                done = !result.MoveNext();

            } catch(Exception) {

                break;

            }


            if(done)

                yield break;

            else

                yield return result.Current;

        }

    }

}

Also there might be a way to keep track of the current & next in the input sequence (and avoid the Skip) but am algorithmically challenged.

Ajai

Max Uvw

Sorry for formatting :(

        public static IEnumerable

<t RobustEnumerating <t (

                    IEnumerable

<t input, Func <ienumerable<t , IEnumerable func)

        {

            return input.

                    Select(i => TryApply(One(i), o => func(o).First())).

                    Where(m => m != Maybe

<t .Nothing).

                    Select(m => m.Value);

        }


        private static IEnumerable

<t One <t (T value)

        {

            yield return value;

        }


        private static Maybe

<r TryApply <t,> (T input, Func <t,> func)

        {

            try {

                return Maybe

<r .Just(func(input));

            } catch {

                return Maybe

<r .Nothing;

            }

        }


        private sealed class Maybe

<t
{

            public static readonly Maybe

<t Nothing = new Maybe <t ();

            public T Value { get; private set; }

            public static Maybe

<t Just(T value)

            {

                return new Maybe

<t {Value = value};

            }

        }
Demid

argggh, the site engine ate template parameters in my functions so it wont compile straight away but I'm sure it's easy to fix.

Dmitry

Here is a version that works with multiple results:

    public static IEnumerable<T> RobustEnumerating<T>(IEnumerable<T> input, Func<IEnumerable<T>, IEnumerable<T>> func)

    {

        foreach (var item in input)

        {

            bool valid;


            IEnumerable<T>results = new T[0];


            try

            {

                results = func(new[] { item }).ToArray();

                valid = true;

            }

            catch

            {

                valid = false;

            }


            if (valid)

            {

                foreach (var result in results)

                {

                    yield return result;

                }

            }

        }

    }
Alex Yakunin

The original problem is easy to solve with interactive extensions from Rx Framework:

Instead of:

foreach (int i in RobustEnumerating(Enumerable.Range(0, 10), FaultyFunc))

use:

foreach (int i in RobustEnumerating(Enumerable.Range(0, 10), FaultyFunc).OnErrorResumeNext())

Nick Aceves

I have a feeling I know why you need this...

I'm guessing that this relates to building indexes, and that you want to be able to iterate over the LINQ query that generates the index, skipping over items that (for some reason or another) cause an exception to be thrown...

If that's right, then you might try this: http://pastie.org/853146

Basically, I just reimplemented the most basic LINQ operators to swallow exceptions when enumerating. If you ditch System.Linq and just include your own implementation of those methods when you compile the index LINQ queries, you should be able to accomplish this.

Nick Aceves

@Alex

I don't think that's what OnErrorResumeNext does. OnErrorResumeNext takes an enumerable of enumerables, and iterates over each one of their members in sequence. If one of enumerables throws, it moves to the next enumerable and starts iterating over its items. That doesn't allow you to iterate over a single enumerable in the way Oren described...

Ayende Rahien

Steve,

What if the func return a different number of elements?

You code with either lose some elements or fail itself when there are no elements

Ayende Rahien

timoconnell ,

Huh?

What happen if I give you a data source over dates?

Ayende Rahien

Mike,

Nice!

Now what happen if I pass in an enumerable that isn't using yield?

Or a different enumerable where state might be something else?

Ayende Rahien

Nick,

I have three different implementations that work :-)

Ayende Rahien

Mike,

What if you can only traverse the enumerable once?

Ayende Rahien

Jay,

This doesn't work with the function provided.

Those enumerables are stateful, and will stop on the first exception

Ayende Rahien

Venu,

The function cannot have any knowledge of the methods I pass to it.

Ayende Rahien

Nick,

Very good observation!

You can still do a lot with it, though.

See my next few posts

Ayende Rahien

Bobby,

You iterate over the source enumerable more than once

Ayende Rahien

Demid,

You hit my second implementation right on the money, but did it much more elegantly!

Nice work

Ayende Rahien

Ajai,

You are iterating over the input enum more than once

Ayende Rahien

Nick,

You are correct about how I run into this.

Wait for the next post on the subject, though, I'll give it full coverage.

And it looks like Linq actually make this easier

Nick Aceves

Oren,

I bet for each one of your implementations, I can devise an enumerator implementation for which they fail ;)

I really don't think this can work in a perfectly general sense (that is, for any enumerator implementation); assumptions have to be made as to the behavior of the enumerator under certain conditions.

But, if you're expecting to get compiler-generated enumerators, or enumerators from LINQ chains, then you can, indeed, make assumptions about how they work.

Kenneth
    public static IEnumerable

<t RobustEnumerating <t(

        IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

    {

        var dd = input.Where(x =>

        {

            string s = x.ToString();

            int i = 0;

            if (int.TryParse(s, out i))

            {

                if (i % 2 == 0) return false;

                return true;

            }

            return false;

        });

        // how to do this?

        return func(dd);

    }
Koen

Pretty sure this is not what you want but it works:

return func(input.Cast <int().Where(i => i % 2 == 1).Cast <t());

Ayende Rahien

Kenneth,

You can't assume the same enumerating function.

Benny Thomas

Ugly code that needs some tuning.

public static IEnumerable <t RobustEnumerating <t(

    IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

{


    foreach(T a in input)

    {

        List

<t results = new List <t();

        try

        {

            results.AddRange(func(new List

<t {a}));

        }

        catch

        {

            // Do some faulthandling here

        }

        foreach (var result in results)

        {

            yield return result;

        }

    }        

}
Alex Yakunin

@ Nick Aceves

You're right - I really made a mistake. Btw, such OnErrorResumeNext is fitting very well on their sequence concept (throw terminates a sequence).

So now it looks like a tricky case to implement this fully on default LINQ combinators and Rx.

Koen

My generic type parameters weren't automatically HTML encoded so here's a second attempt encoding them myself:

return func(input.Cast().Where(i => i % 2 == 1).Cast());

Sergey Kachkovsky

public static IEnumerable <t RobustEnumerating <t(

        IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

    {

        IEnumerator

<t enumerator = func(input).GetEnumerator();

        do

        {

            bool valid = true;

            try

            {

                if (!enumerator.MoveNext()) yield break;

            }

            catch

            {

                valid = false;

            }

            if (valid)

                yield return enumerator.Current;

        } while (true);

    }
Mike Doherty

Ok. Here is my solution:

public static IEnumerable

<t RobustEnumerating <t (

    IEnumerable

<t input, Func <ienumerable<t , IEnumerable func)

{

    var rowsToSkipOnError = 1;

    var enumerationToWrap = func(input).GetEnumerator();

    while (true)

    {

        T nextValue;

        try

        {

            if (!enumerationToWrap.MoveNext())

                yield break;


            nextValue = enumerationToWrap.Current;

        }

        catch (Exception)

        {

            // This input failed so skip to the next one

            input = input.Skip(rowsToSkipOnError);

            enumerationToWrap = func(input).GetEnumerator();

            rowsToSkipOnError = 1;

            continue;

        }


        rowsToSkipOnError++;

        yield return nextValue;

    }

}
Arielr

return input.SelectMany(x =>

                        {

                            try

                            {

                                return func(x.AsEnumerableX()).ToArray();

                            }

                            catch

                            {

                                return Enumerable.Empty

<t();

                            }

                        });
Mike Doherty

This time I've escaped the angle brackets to see if that slips past the comment sanitizer.

Looking over Dimid's solution though I realize that I am repeatedly enumerating over the input which could have a performance impact if getting these values was expensive. I love the way that he wraps the input to preserve it's state.

public static IEnumerable<T> RobustEnumerating<T>(

   IEnumerable<T> input, Func<IEnumerable<T>, IEnumerable<T>> func)

{

   var rowsToSkipOnError = 1;

   var enumerationToWrap = func(input).GetEnumerator();

   while (true)

   {

       T nextValue;

       try

       {

           if (!enumerationToWrap.MoveNext())

               yield break;


           nextValue = enumerationToWrap.Current;

       }

       catch (Exception)

       {

           // This input failed so skip to the next one

           input = input.Skip(rowsToSkipOnError);

           enumerationToWrap = func(input).GetEnumerator();

           rowsToSkipOnError = 1;

           continue;

       }


       rowsToSkipOnError++;

       yield return nextValue;

   }

}
Frank

Without having looked at previous posted entries, this should do the trick, while still supporting infinite enumerations:

public static IEnumerable <t RobustEnumerating <t
(

IEnumerable

<t input,

Func

<ienumerable<t, IEnumerable func

)

{

var singleInputElementArray = new T[1];

foreach (var inputElement in input)

{

    singleInputElementArray[0] = inputElement;


    IEnumerator

<t enumerator = func(singleInputElementArray).GetEnumerator();

    bool success;

    do

    {

        success = false;


        try

        {

            if (enumerator.MoveNext())

                success = true;

        }

        catch

        {

        }


        if (success)

            yield return enumerator.Current;

    }

    while (success);

}

}

Sergey Shishkin
        return input.SelectMany(x =>

        {

            try

            {

                return new[]{func(new[] { x }).FirstOrDefault()};

            }

            catch

            {

                return new T[0];

            }

        });
tobi

I cannot believe that 95% of all answers are just wrong. It is not that hard... If your call to movenext is not in a try-catch, your code is wrong. if you use foreach, your code is wrong because an exception will skip the rest of the enumeration. Do not call func multiple times, it is not necessary. It will enumerate the sequence multiple times.

Nice challenge however.

Mike Thomas

@ayende or if MoveNext just sets some other flag before it throws or if MoveNext disposes of itself before it throws or if MoveNext does something like this (assuming for the moment that I still reset the state correctly):

yield return memberCounterThatINeedToBeUniquePerIncomingItem + transformThatWontThrow();

yield return memberCounterThatINeedToBeUniquePerIncomingItem + transformThatWillThrow();

memberCounterThatINeedToBeUniquePerIncomingItem++;

Basically, like Nick says, if we have the requirement that the func can be called only once (which makes sense since it could be keeping track of its position or could make a million expensive remote calls before returning the first item) you can't solve this in a generic way.

tobi

the func transforms a non-throwing sequence into a throwing sequence. you can solve it calling it only once.

Michal

return input.Select(i =>

                    {

                        try

                        {

                            return func(new[] { i }).First();

                        }

                        catch

                        {

                            return default(T);

                        }

                    }).

                Where(i => !i.Equals(default(T)));
Ajai Shankar

@Ayende "Ajai, You are iterating over the input enum more than once"

Here is what I think works (I had to add a method, don't know if it is allowed)

Very nice problem, do you expect the input enumerable to throw too?

Now I need to go get some real work done :-)

Looking forward to what you have, can't take the suspense any longer!

static IEnumerable

<t EnumOnce <t(IEnumerator <t e)

{

    while(e.MoveNext())

        yield return e.Current;

}


public static IEnumerable

<t RobustEnumerating <t(

    IEnumerable

<t input,Func <ienumerable<t, IEnumerable func)

{

    input = EnumOnce(input.GetEnumerator());


    var result = func(input).GetEnumerator();

    while(true) {

        try {

            if(!result.MoveNext())

                break;

        } catch(Exception) {

            result = func(input).GetEnumerator();

            continue;

        }

        yield return result.Current;

    }

}
Kvasov Roman

I think its more robust implementation with additional class, but it call function only once so if it contains some initialization logic it doesnt break.

using System;

using System.Collections;

using System.Collections.Generic;

using System.Linq;


public class Program

{

    public class StatefullEnumerable

<t : IEnumerable <t {

        private IEnumerator

<t enumerator;

        public StatefullEnumerable(IEnumerable

<t enumerable) {

            enumerator = enumerable.GetEnumerator();

        }


        public IEnumerator

<t GetEnumerator() {

            return enumerator;

        }


        IEnumerator IEnumerable.GetEnumerator() {

            return GetEnumerator();

        }

    }


    private static void Main()

    {

        foreach (int i in RobustEnumerating(Enumerable.Range(0, 10), FaultyFunc))

        {

            Console.WriteLine(i);

        }

    }


    public static IEnumerable

<t RobustEnumerating <t (

        IEnumerable

<t input,Func <ienumerable<t , IEnumerable func)

    {

        var enumerable = func(new StatefullEnumerable

<t (input));

        var enumerator = enumerable.GetEnumerator();

        while (true) {

            var fail = false;

            var value = default(T);

            try {

                if (!enumerator.MoveNext())

                    yield break;

                value = enumerator.Current;

            }

            catch (Exception) {

                fail = true;

                enumerator = enumerable.GetEnumerator();

            }

            if (!fail)

                yield return value;

        }

        // how to do this?

    }


    public static IEnumerable

<int FaultyFunc(IEnumerable <int source)

    {

        foreach (int i in source)

        {

            yield return i/(i%2);

        }

    }

}
tobi

Wow I will remember this challenge and use it as an interview question. It seems like everyone gets it wrong the first time so you can discuss the problem with the candidate to see if they are smart or not.

Also I do not understand why everyone seems to restart the sequence when an error has occurred. Just do nothing or else the sequence will restart over and over.

I do not understand the purpose of StatefullEnumerable. It seems to do nothing.

James C-S

This is probably not in the spirit of the challenge, but ...

public static IEnumerable

<t RobustEnumerating <t(

    IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

{

    return func(input.Cast

<int().Where(x => x % 2 != 0).Cast <t());

}

...works just fine. Of course it does break encapsulation. :-)

James C-S

OK. I felt bad about me previous answer that broke encapsulation. Here's an answer that doesn't.

    public static IEnumerable

<t RobustEnumerating <t(

        IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

    {

        Func

<ienumerable<t, IEnumerable safeFunc = source =>

        {

            var @return = (IEnumerable

<t)null;

            try { @return = func(source).ToArray(); }

            catch (System.DivideByZeroException) { }

            return @return;

        };

        return (from i2 in

                    (from i in input

                     select new T[] { i })

                let o2 = safeFunc(i2)

                where o2 != null

                select o2).SelectMany(o => o);

    }
James C-S

Whoops! Now I feel bad about me, err my, grammar... ;-)

Demid

I can't believe so many people are so lazy to compile their code at least.

It's not a crime do not know that try/catch block isn't compatible with yield statement but you can at least try to compile it.

Download SnippetCompiler. It is outdated project but it really really good to test your ideas whenever you're not sure how compiler or .net behaves in a particular situation because. It loaded instantly and I ran Oren's example in two seconds after I saw it.

Dmitriy Nagirnyak

@Demind, I use LINQPad for that purpose. Copy-paste code and it just runs.

Ajai Shankar

@Demid & Dmitriy: vi & csc works too :-)

Ken Tong

How about this?

public static IEnumerable<T> RobustEnumerating<T>(

    IEnumerable<T> input, Func<IEnumerable<T>, IEnumerable<T>> func)

{

    IEnumerable<T> results = new T[0];


    foreach (T item in input)

    {

        try

        {

            foreach (T value in func(new[] { item }))

                results = results.Concat(new[] { value });

        }

        catch { }

    }


    foreach (T result in results)

        yield return result;

}
Allan PArker

Hi,

Here is my attempt at this:

    public static IEnumerable

<t RobustEnumerating <t(IEnumerable <t input, Func <ienumerable<t, IEnumerable func)

    {

        foreach (T item in input)

        {

            IEnumerable

<t result;

            var list = new List

<t { item };

            try

            {

                result = func(list);

                var test = result.First();

            }

            catch

            {

                continue;

            }


            yield return result.First();

        }

    }
Thomas Eyde

It looks like the solution is a variation over iterating the input, because that's the only thing we can control, evaluate each item and only return the results of those evaluations that don't fail.

Here's my attempt:

            foreach (var item in input)

            {

                T[] evaluatedItems;

                try

                {

                    var itemAsEnumerable = new[] { item };

                    evaluatedItems = func(itemAsEnumerable).ToArray();

                }

                catch (Exception)

                {

                    continue;

                }


                foreach (var evaluated in evaluatedItems)

                {

                    yield return evaluated;

                }

            }
Chris Martin

public static IEnumerable <int RobustEnumerating(IEnumerable <int input, Func <ienumerable<int, IEnumerable func)

{

return func(input.ToList().ConvertAll(t => int.Parse(t.ToString())).Where(i => i%2 > 0));

}

lol

firefly

@ tobi, it's not hard but I wouldn't say it's intuitive either. Especially for an interview question unless you've wrote library code where you use MoveNext a lot and have an intimate understanding of how it really work.

Now if you've been exposed to this before somewhere else then it's extremely easy :). Thanks Oren and everyone else that comments. I learned a lot from the exposure... especially from Demid code, very elegant indeed.

sergio

Maybe its not the best, but it works

public class Program

{

    private static void Main()

    {

        foreach (int i in RobustEnumerating(Enumerable.Range(0, 10), FaultyFunc))

        {

            Console.WriteLine(i);

        }

        Console.ReadLine();

    }


    public static IEnumerable

<t RobustEnumerating <t(

        IEnumerable

<t input, Func <ienumerable<t, IEnumerable func)

    {

        // how to do this?

        var enumerator = func(input).GetEnumerator();

        while (SafeMoveNextEnumerator(input, func, ref enumerator))

        {

            yield return enumerator.Current;

        }




    }

    static int skip = 0;


    public static bool SafeMoveNextEnumerator

<t(IEnumerable <t input, Func <ienumerable<t, IEnumerable func, ref IEnumerator <t safe)

    {

        while (true)

        {

            skip++;

            try

            {

                return safe.MoveNext();

            }

            catch

            {

                var e = Enumerable.Skip(input, skip);

                safe = func(e).GetEnumerator();

            }

        }

    }


    public static IEnumerable

<int FaultyFunc(IEnumerable <int source)

    {

        foreach (int i in source)

        {

            yield return i / (i % 2);

        }

    }

}
James

Simplest possible solution ...

    public static IEnumerable

<int RobustEnumerating <t(IEnumerable <t input, Func <ienumerable<t, IEnumerable func)

    {

        return new[] { 1, 3, 5, 7, 9 };

    }

Comment preview

Comments have been closed on this topic.

FUTURE POSTS

  1. Concurrent max value - 14 hours from now
  2. Production postmortem: The case of the memory eater and high load - 4 days from now
  3. Production postmortem: The case of the lying configuration file - 5 days from now
  4. Production postmortem: The industry at large - 6 days from now
  5. The insidious cost of allocations - 7 days from now

And 5 more posts are pending...

There are posts all the way to Sep 10, 2015

RECENT SERIES

  1. Find the bug (5):
    20 Apr 2011 - Why do I get a Null Reference Exception?
  2. Production postmortem (10):
    14 Aug 2015 - The case of the man in the middle
  3. What is new in RavenDB 3.5 (7):
    12 Aug 2015 - Monitoring support
  4. Career planning (6):
    24 Jul 2015 - The immortal choices aren't
View all series

Syndication

Main feed Feed Stats
Comments feed   Comments Feed Stats