Asynch Iteration

Okay - so the title's a bit misleading, but I thought it might peak interest :-)  At Win-Dev this week, Frank Redmond asked the track chairs and some Microsoft folks to come up with a few short programming puzzles that he could use to raffle off some swag.  As the chair for the CLR/C# track, I submitted a few.  We didn't have room for all of the submissions, so I thought I'd post one that didn't make it up on a whiteboard here.  It has got to be one of the ugliest pieces of code I've ever written.

What does the following program display?  Explain your answer (an answer w/o an accurate explanation doesn't count).  Compile with your eyes only.

using System;
using System.Collections;
using System.IO;

class Quiz {
  static void Main() {
   
foreach( char c in CO("Hello, world!") ) {
     
Stream.Null.BeginWrite(
       
new byte[]{0x21}, 0, 1,
       
delegate(IAsyncResult r) {
         
Console.Write(r.AsyncState);
         
Stream.Null.EndWrite(r);
       
},
       
c
     
);
   
}
   
Console.ReadLine();
 
}
 
static IEnumerable CO(string s) {
   
foreach (char c in s.ToCharArray()) {
     
yield return c;
   
}
 
}
}


Posted Oct 27 2004, 04:10 AM by mike-woodring

Comments

Luc Cluitmans wrote re: Asynch Iteration
on 10-27-2004 7:01 AM
Well, as far as I can deduce without compiling it, this is just a fancy way of writing "Hello, world!" to the console (without a trailing newline...).

The 'foreach' in Main() is equivalent to 'foreach( char c in "Hello, World!")', iterating over the letters in the string (the iterator CO is just a fancy way to do that).

Inside the loop, a byte with value 0x21 is written to System.Null for each input character (which disappears into the bit bucket), but the completion method (the anonymous delegate) writes the item that was passed as asynchronous state to the console. That item is the character from the string. So the result is that all characters from "Hello, World!" are written to the console. I assume (without compiling) that they are written in their original order...
Luc Cluitmans wrote re: Asynch Iteration
on 10-27-2004 7:02 AM
One additional note:

the Console.ReadLine is there to ensure that the writes actually can complete before the program exits...
Mike wrote re: Asynch Iteration
on 10-27-2004 7:18 AM
Correct so far (especially the follow up about Console.ReadLine :-).

But if BeginWrite is being used to issue async writes to the null stream, then why are the write-completion callbacks to my anonymous delegate happening in the correct order?
Chris Sells wrote re: Asynch Iteration
on 10-27-2004 7:20 AM
The yield statement in the static CO method causes C# to build an enumerator that hands out each character of the string 's' passed as an argument to the method.

The call to CO in the foreach causes the creation of an enumerator for the string "Hello, world!".

For each character, a '!' is written asynchronously to the Null stream (which goes nowhere, but doesn't block doing it).

When the asynchronous write is completed, the anon delegate is called that writes the character of the string to the Console (passed as the last argument to the BeginWrite call as the AsynchState argument) and ends the asynchronous write to the Null stream.

When the string has been written, the programs waits for the Enter key without first writing a CR/LF (which is good, since it could get written at any random place in the string due to the asynchronous nature of the app unless it was tacked onto the end of the string being enumerated). The short-hand equivalent of this app is the following:

Console.Write("Hello, world!");
Console.ReadLine();
Stephen Toub wrote WinDev Coding Questions
on 10-27-2004 8:13 AM
Stephen Toub wrote WinDev Coding Questions
on 10-27-2004 8:17 AM
Luc Cluitmans wrote re: Asynch Iteration
on 10-27-2004 9:30 AM
Quote:
"But if BeginWrite is being used to issue async writes to the null stream, then why are the write-completion callbacks to my anonymous delegate happening in the correct order?"

Eh, one explanation would be that the implementation behind Stream.Null uses the standard Stream.BeginWrite implementation, which is documented (at least in the Whidbey beta1 docs) as actually being a wrapper around Write(), in other words: the call is actually synchronous, not asynchronous...

And a little check with Lutz Roeder's great 'Reflector' tool confirms that the class System.Stream.NullStream (a private inner class of System.Stream), which implements Stream.Null, indeed does not override BeginWrite()...
Mike wrote re: Asynch Iteration
on 10-27-2004 9:49 AM
Bingo. I should have mentioned using Reflector, Anakrino, or ILDASM was against the rules too :-)
Scott Allen wrote re: Asynch Iteration
on 10-27-2004 11:53 AM
It's harder to obfuscate without a good C++ preprocessor in the compilation mix :)
Ernst Kuschke wrote re: Asynch Iteration
on 10-27-2004 12:41 PM
Even if there was an override for BeginWrite, the letters would have come out in the same order ;o)
When EndInvoke is called, the main thread blocks until the async call is complete. This means that the previous async call *will* have completed by the time the next async call is invoked.
Chris Taylor wrote re: Asynch Iteration
on 10-27-2004 1:42 PM
Ernst, the EndWrite is called in the delegate not the main thread therefore the main thread would not block.
Matthias Ernst wrote re: Asynch Iteration
on 10-27-2004 1:45 PM
Hmm. Stream#beginWrite is synchronous BUT it does invoke the callback asynchronously via beginInvoke. The thread executing main is a different one from the one executing the callback. Why do all these asynchronous invocations happen in the right order ? I'm still suspecting we're just lucky that the CLR thread pool schedules them in only one thread.

Or am I getting something terribly wrong?
Ernst Kuschke wrote re: Asynch Iteration
on 10-28-2004 3:17 AM
Hi Chris, the mainthread would indeed block. EndInvoke blocks the mainthread until the async call has completed.
Try the following, and break on the indicated lines to check the order of calls made:

delegate string delegateFoo(string c);

public static void Main()
{
delegateFoo d = new delegateFoo(Foo);

for(int i = 0; i <= 5; i++)
{
IAsyncResult ar = d.BeginInvoke(i.ToString(), null, null); //Break here
d.EndInvoke(ar); //Break here
Console.WriteLine("Done with call " + i.ToString()); //Break here
}

Console.WriteLine("Done!");
Console.ReadLine();
}

private static string Foo(string p)
{
string s = "!!!";
Thread.Sleep(10000);
return (s += p);
}
Mike wrote re: Asynch Iteration
on 10-28-2004 7:18 AM
But that's not an accurate representation of what my sample code was doing.

In the sample, the anonymous delegate that points to the function which calls EndInvoke is being passed to Stream.BeginWrite. The implementation of Stream.BeginWrite uses the async Delegate.BeginInvoke to call that delegate (easy enough to see using Reflector - look at the last line of code in BeginWrite). So although the write to the Null stream (of 0x21) is occurring synchronously in the context of the main thread, the call to my completion callback (which writes to the console) is occurring asynchronously in the context of a thread pool thread.

So why is the output still correctly sequenced?
Matthias Ernst wrote re: Asynch Iteration
on 10-28-2004 8:43 AM
Do we have a version problem here? 2.0.3600 of mscorlib shows how Stream.write does not synchronously write, but serializes all output *and* the callback through a semaphore:

[HostProtection(SecurityAction.LinkDemand, ExternalThreading=true)]
public virtual IAsyncResult BeginWrite(byte[] buffer, int offset, int count, AsyncCallback callback, object state)
{
if (!this.CanWrite)
{
__Error.WriteNotSupported();
}
Stream.WriteDelegate delegate1 = new Stream.WriteDelegate(this.Write);
if (this._semaphore == null)
{
lock (this)
{
if (this._semaphore == null)
{
this._semaphore = new Semaphore(1, 1);
}
}
}
bool flag1 = this._semaphore.WaitOne();
this._writeDelegate = delegate1;
return delegate1.BeginInvoke(buffer, offset, count, callback, state);
}

I ported your code to 1.1 and if you write a little more text, after a while the threadpool will expand and the output is no longer correctly sequenced.

So "Compile with your eyes only" was actually a pretty difficult entertainment ...
Mike wrote re: Asynch Iteration
on 10-29-2004 6:38 AM
I realized this on the plane home last night. I'd run everything on the 2.0 runtime but inadvertently used reflector against the 1.1 binaries to peek at the code.
Some Assembly Required wrote Fun with Anonymous Delegates & Lambda Expressions
on 06-30-2008 3:24 PM

During the course of teaching a customized offering of .NET Framework Fundamentals last week, I was showing

Add a Comment

(required)  
(optional)
(required)  
Remember Me?