Month: May 2004

What are Closures?

There has been a lot of talk recently hearting up the wires in cyberworld on Closures and c#. One of the question that some have asked is what the heck are closures? Well I don’t know what all the excitment is all about, but closures are nothing but what is also called the Anonymous Methods. There now that you know it, go do something productive! BTW, anonymous methods are cool and with the combination of iterators you can go pretty interesting things. Check out the latest msdn article for a quick read.

Managed object lifetime – How a Garbage Collector Works

This is part 1 (if you will) of a couple of posts. I am surprised how very little most people know how the GC actually works. If you are reading this and are going to be interviewed by me, this is one of my favourite questions. *grin* This is based on an interesting post I found long ago I think on gotdotnet (sorry I forgot who the author was, I’ll try and google it and see if I can find the link to the original post).

 

It is not possible to state exactly when a managed object will be collected. The garbage collector schedules itself based on various heuristics. Even if a garbage collection occurs, it may only collect the younger generations of the heap. And the JIT has some freedom to lengthen or shorten the lifetime of instances, based on how it generates code and reports liveness.

1class C {

2   IntPtr _handle;

3   Static void OperateOnHandle(IntPtr h) { ... }

4   void m() {

5      OperateOnHandle(_handle);

6      ...

7   }

8   ...

9}

10

11class Other {

12   void work() {

13      if (something) {

14         C aC = new C();

15         aC.m();

16         ...  // most guess here

17      } else {

18         ...

19      }

20   }

21}

So we cannot say how long “aC” might live in the above code (line 15). The JIT might report the reference until Other.work() completes. It might inline Other.work() into some other method, and report aC even longer. Even if you add “aC = null;” after your usage of it, the JIT is free to consider this assignment to be dead code and eliminate it. Regardless of when the JIT stops reporting the reference, the GC might not get around to collecting it for some time.

You should worry more about the earliest point that aC could be collected. If you are like most people, you’ll guess that the soonest aC becomes eligible for collection is at the closing brace of Other.work()’s “if” clause (line 16). The fact though is, that braces don’t exist in the IL; they are a syntactic contract between you and your language compiler. Other.work() is free to stop reporting aC as soon as it has initiated the call to aC.m().

 

Another common guess is that the soonest aC could be collected is when C.m() stops executing. Or perhaps after the call to C.OperateOnHandle(). Actually, aC could become eligible for collection before C.m() even calls C.OperateOnHandle(). Once we’ve extracted _handle from “this”, there are no further uses of this object. In other words, “this” can be collected even while you are executing an instance method on that object.

 

Why should you care? For the example above, you don’t; the GC’s reachability will ensure that objects won’t be collected until we are finished with them. But what if class C has a Finalize() method which closes _handle? When we call C.OperateOnHandle(), we now have a race between the application and the GC / Finalizer – eventually we’re going to lose that race.

 

One way to fix this race is to add a call to GC.KeepAlive(this) right after the call to OperateOnHandle(). This indicates that we need the JIT to keep reporting “this” to the GC until we get to that point in the execution. KeepAlive is just a light-weight method call that is opaque to the JIT. So the JIT cannot inline the call and recognize that the call has no real side effects and hence could be eliminated. The reason you need to add this call is that you have really broken the encapsulation of the _handle resource. The lifetime of the enclosing object and the required lifetime of the _handle are separated when you extract the value from the object’s field.

 

It’s bad enough that you must use GC.KeepAlive() to tie those two lifetimes back together in your encapsulation. It would be disastrous if you required the clients of your class to be responsible for calling KeepAlive. Public fields on classes are a bad idea for many reasons. As we’ve seen, when they expose a resource that is subject to finalization, they are an exceptionally bad idea.

 

You may wonder why we don’t just extend all lifetimes to the end of methods. This has a terrible impact on code quality, particularly on x86 where we are cursed with limited registers. And a change like that doesn’t really fix the problem. It is still possible for you to return the _handle, place it in a static field, or otherwise cause its lifetime to escape the lifetime of the enclosing object.

 

There’s another wrinkle to this issue. So far we’ve seen how the Finalizer thread and the application can race when the resource can be separated from its enclosing object. The same sort of thing can happen when you expose IDisposable on your class. Now a multi-threaded application can simultaneously use the resource on one thread and imperatively call Dispose on another thread. GC.KeepAlive isn’t going to solve this problem, since you’ve provided a public API to disassociate the lifetime of the resource from the lifetime of the enclosing object.

 

This is more than application issue. It can also be used to mount security attacks. If malicious code can open a file to an uninteresting part of the filesystem, it could simultaneously Read and Dispose that file object on two different threads. In a server environment, it’s possible that some other component is opening a file to a sensitive part of the filesystem. Eventually, the malicious code could exploit the race condition to read the other component’s file. This is a handle-recycling attack.

 

When the .NET framework uses a resource via a PInvoke to the OS (like reading from a file handle), it places a reference count on the resource. If malicious (or poorly-timed) code calls Dispose, this simply removes the reference count that was created when the resource was acquired. The result is that all current uses of the resource will be drained, the resource will then be safely disposed, and subsequent attempts to use the resource will be failed gracefully.

 

For now, you should consider similar approaches if you are encapsulating sensitive resources like this, which are subject to recycling. Also, this probably should be the second post, I’ll probably change it the title (or repost) to avoid confusion.

IronPython: A fast Python implementation for .NET (and Mono?)

Karan (see the Blogs I read section for his weblog) pointed me to this and I think it is pretty cool. IronPython is a new implementation of the Python language targeting the CLR. It compiles python programs into IL that can run on either .NET or Mono (now that is cool). This produces a standard assembly that other code can reference. It would be an interesting exercises to see the IL it produces and see if we can infer anything from that.

IronPython can run fast on .NET. Python is an extremely dynamic language and this offers compelling evidence that other dynamic languages should be able to run well on this platform. However, implementing a dynamic language for the CLR is not a simple process. The CLR is primarily designed to support statically typed object orienter and procedural languages. Allowing a dynamic language to run well on this platform requires careful performance tuning and judicious use of the underlying CLR constructs. You can read more on the perf. results of this here.

Cracking the Da Vinci Code

First of all, if you don’t know what the hell am I talking about, jeez, where have you been buddy? If you do but have not read the book, then please do yourself a favour and do so. You can buy it from here. Once you do finish reading the book, then come back here and we can talk about it.

The crux of the book relies on the golden ratio also known as the divine proportion which has a value of 1.61803399 (per the book). The book, claims an eerie prevalence of divine proportion  in life and nature, which if you think is just fiction might be surprised to find out it really is not!! Whaaaaaat? Dammit, and all this time I thought I was reading just a novel!! Err.. I guess I should stop trying to be a “writer“ and get on with it.

I think some of the truth is a bit stretched in the novel, but nonetheless based on truth (of course IMHO).The Divine Proportion (represented by the Greek letter f – phi). In reality the value of phi is:

f = (1 + v5) / 2

If you actually try and solve the above, you will figure out that it is an irrational number i.e. the decimal expansion of an neither terminates or repeats.

 

The whole thing goes back to the Greeks, who believed the most pleasing and aesthetic and purest form is the triangle, specifically the golden triangle, which the Greeks incorporated in their architecture. The Italian mathematician Leonardo Pisano’s book called Liber Abaci brings to the west a  method of writing numbers and arithmetic (that we use today) which was completed in India 500 years earlier. These new numbers eventually provided the basis for modern science and engineering. One of the many exercises in Liber Abaci leads to the Fibonacci series (1, 1, 2, 3, 5, 8, 13, 21, …). I won’t go into the examples here but you can find the link in the bottom of this post. The Fibonacci series essentially is every one number after 1 is the sum of the previous two (so, 1+1 = 2, 1+2 = 3, 2+3 = 5, etc.). As you solve more problems (the rabbit problem, bees, etc. – again see the link at the end) with the Fibonacci series you will notice that this series occurs pretty frequently in nature. Some examples (also mentioned in the book) are:

  • If you count the number of petals in most flowers you will find the total to be a Fibonacci series. E.g. an iris has 3 petals, a buttercup has 5, a delphinium 8, daisy 13, 21 or 34, etc.
  • A sunflower has two beautiful spiral patterns – one clockwise and the other anti-clockwise. If you count those spirals, you will find there are 21 or 34 running clockwise and 34 or 55 running anti-clockwise.
  • Similarly, other flowers exhibit the same characteristics including Pine-cones and Cauliflowers
  • Leaves on trees (and stems) are arranged in a spiral pattern that is wound around the stem. If p is the number of turns you make (of the spiral) till you reach the next leaf directly above the first, and q is the number of leaves (excluding the first one), then the ratio p/q gives us the divergence of the plant. If you calculate the divergence for various plant species you will find that both the numerator and denominators represent Fibonacci series – 1/2, 1/3, 2/5, 3/8, 5/13, 8/21, etc.

Here is where things get more interesting. If you take the Fibonacci series and divide each number into the one that follows it you would get: 1/1 = 1; 2/1 = 2; 3/2 = 1.5; 5/3 = 1.666; 8/5 = 1.6; 13/8 = 1.625; 21/13 = 1.615; 34/21 = 1.619; 55/34 = 1.6176; 89/55 = 1.6181…. Hmmmm, notice anything? The 1, 1.6, 1.61, 1.618, etc. looks like the golden ratio.

 

Mathematic ans have proved that the Fibonacci series gets slowly closer to f and finally equate at infinity.

 

What does this mean? Well I cannot say for all the stuff laid out in the book regarding the golden ratio versus the human body (e.g. measuring the distance from the tip of your head to the floor and then dividing the distance from your belly button to the floor – did you get a f or something around that number?), but the facts for plants is there in front of us and such frequent appearances cannot be accidental.

 

So what do you make of all this? Well we found out quite recently that it is a matter of efficiency. To achieve maximum efficiency, flower heads and plant leaves grow in a spiral fashion governed by the golden ratio and rounded off to the nearest whole number (since f is a irrational number) – because of this it will be a Fibonacci series.

 

Why spirals and the golden ratio? Well as new leaves are added, they need to be the least obstructive to the old leaves and the new ones (that would come later) above it. To do so, the leaves come in a spiral pattern. In the case of seeds, a spiral “packs“ the most possible – and the most efficient way to do is via spirals. We now understand that f is the ratio that gives the optimal solution to growth equations. The mathematical explanation is that of all the irrational numbers, f is in a very precise (and technical sense) farthest from being representable as a fraction.

 

So, as a good novel should, this one brings the plot to an end satisfactorily and also leaves one wondering about life. Also makes one wonder how much of todays religions teachings are fact and why are people so willing to accept things at so much face value? Why are we so eager to believe in everything we are told?

 

To conclude as it is in the book, the “Phi (f) is one H cooler than Pi (p)”.

 

Check out the links here and here which formed the basis of this post.

 

Update: I am not sure but the symbol for Phi shows up as a “f” and the Pi shows up as a “p”. I will see if I can fix that, but till then, whenever you see just a “f” or a “p”, just read it as Phi and Pi respectively.

Improving .NET Application Performance and Scalability

PAG group just released the new library. If you have been playing with the beta of this, you know there is lots of good stuff. A excellent reference to keep near you! Now that this is “public” I will be posting some of the things I find here – watch this space, and till then, read up and enjoy!

Installing Whidbey on Longhorn [Build 4074]

Miguel Jimenez has a post on how to install Whidbey on the new Longhorn bits. There are some comments on some problems. I myself have not tried this (yet), will be posting my first impressions on the new build and then try this after that – probably this weekend or early next week.

Iterators in c#

I have been meaning to write something on what are iterators, well actually more on how they work and why is there so much “hype” around them, the only problem being the best excuse we all can fall back to – Time *grin*. After all Java already has them, so is that a big deal that .NET would (when Whidbey is released) have them. Anyways, the point being that Roshan has a decent post on that already – probably a much better job that I would do.

BTW, the Java implementation is a bit different on how Whidbey is going to proceed (as of now), but the next version of the J2EE specs (I think changes it, not sure, any J2EE knowledgeable folks reading this care to comment)?

DARPA is at it again

From the last time, when DARPA issued the grand challenge, the cameras were there, the million bucks were there (drool), the people were there, the hype were there and the robots were there. So what was missing? Well the Robots followed my example and forgot their “brains” and were half blind and stone dumb! So instead of learning from that and letting the existing teams push on they want to try LAGR. As I say, common sense is not very common!

Becoming an Architect

There has been a very interesting (imho) multi-part interview with the author of Beyond Software Architecture where he discussing various aspects like the relationship between Architecture, Culture and Goodness. Also goes on to discuss the social role of an architect, conceptual integrity and the importance of domain knowledge. I particularly like the quote … “ An architecture is like a dirt road. The first time a car drives over it, the road looks about the same. By the time the 10,000th car drives over it, though, there will be grooves in the road. As an architecture mature, it gets grooves.”. Some other interesting responses that I personally have learned from over the years and I can relate to are below.

How do you measure a good Architect? – How well the architecture responds and adapts, and how well it goes through that construction process, is a measure of whether that architecture is any good.

What is the difference between maturity and completeness? – In its first version, an architecture should be like a baby. It is whole and complete. It has ten fingers and ten toes. It has all its systems, but it is immature. You wouldn’t expect it to do certain things that you might expect a mature system to do. Of course, the definition of what is complete and incomplete, mature and immature, is subjective. But it is really important to distinguish between incompleteness and immaturity. I see the people make the mistake made all the time of building an incomplete system. It’s actually not a mistake that’s correlated to a development process, near as I can tell. You can have iterative, agile projects that produce incomplete results, and you can have waterfall or traditional kinds of projects that produce incomplete results.

“… explore alternative architectures to see what works best…” This is sensible advice. It is also virtually useless.

“… An architecture is like a carefully designed garden, and needs care and feeding….”

What is Entropy Reduction? – Entropy reduction means I am going to make whatever compromise I need to make to get the release done, and then I’m going to give the team the opportunity to go back and respond or recover from the compromises they made. It is a very different concept than refactoring. I think of refactoring as non-architectural, but rather very much in the context of normal work.

This is so true of one of my previous projects we were implementing in the New York office of the company I work for. Some of my colleagues who are reading this (and proving they don’t have anything better to do) are probably grinning ear-to-ear from this:
Another aspect of matching architecture to human needs is matching roles and responsibilities of the people building a system to the appropriate parts of the architecture. Some pundits say that everyone likes to do everything, but that’s not true. I suspect that a lot of people who write about architecture, although they are very smart, have never managed large teams. I’ve managed a 60-person team, and I personally know UI guys who would never get near a database, because they think databases are “stupid”. And I know many database developers who would never go near a UI. The last thing these database people want to do is the full architectural stack. They’ve spent their life becoming an expert at Oracle. And you know what? They are an expert at Oracle. They aren’t going to be an expert at UI design, and given what they would normally design for a UI, you wouldn’t want them to be touching the UI anyway. When you look at the stack in your architecture, and then you look at this mass of people on your team, the architecture and the people have to match. Otherwise you’ll have problems.

Determining the Appropriate Granularity of the Design? – When people talk about architecture, there’s rarely a corresponding discussion of the granularity of the architectural metaphors, components, or descriptions that are being employed. The appropriate granularity is a function of the number of people working on the software and the complexity of the software itself. If only two or three people are working on the project, then they can get pretty granular. As the software and team gets bigger in size, the granularity must become more abstract.

Overall, like I opened this post with, this more or less reflects my experiences and pretty much on the dime with the issues, suggestions and analogies discussed. What are your experiences? Do you disagree with something?

Searching beyond google

Recently google has been all over the news – online, print and television, atleast here in the US; and why not, with their IPO and everything, lot of people are hoping this would kickstart the recovery of the economy. I have been toying with a few other search engines for a few months now. These have also been highlighted in the media recently alongwith the google hype as a potential threat to google. The engines I have been toying with apart from google are:

  • http://sandbox.msn.com – only for news
  • www.kartoo.com – try the flash version – very cool
  • www.mooter.com – small Australian company that clusters results into concepts, allowing faster decisions and deeper digging with less clicks
  • www.teoma.com – applies authority, a new measure of relevance, to deliver authoritative search results. To determine the level of authority associated with a particular site, Teoma employs three proprietary techniques: Refine, Results and Resources

Do you have any “new” ones that you use? If you have not heard of the ones above, give them a spin, each has its own pros and cons. You would need to spend a little time on each using it in your daily tasks to really judge which one works better for what things.

Windows vs. Linux (and oh ya Mac too) – Common OS Myths Debunked

This reminds me of a few weeks ago when during one of my IM sessions with my buddy Karan I was bashing up Linux (which from those who know is not really me). This lead to (atleast that is what I think) a frantic call from him – all the way across the pond – and we yapped like a pair of teenage girls for close to an hour +. The crux of that debate… err.. discussion we had was Windows vs. Linux. If memory servers correctly (which ain’t a lot *grin*), then this article pretty much sums it up. Here are some of the common OS myths that typically float around in cyberspace:

  • Linux is the OS that “just works”
  • Windows is Insecure
  • Windows has better h/w support
  • Linux does a few things and does them well
  • Windows is bad for the server
  • Mac is the best since it is as easy to use as windows, and has the stability of UNIX
  • Linux is ready for the desktop

From my perspective I don’t really care. I have not programmed much on Linux compared to Windows, so I am a bit biased towards Windows but I am open to ideas and at the end of the day am a geek. Most people tend to forget that in most businesses IT is purely an overhead and is just looked as an enabler to the business itself, there of course as always are exceptions, but as a general principle it is true. Do what ever works for you in your environment (home/school/work/whatever) and pick and choose things that you are comfortable with or curious to learn about.

ASP.NET Version Switcher

This utility that can be used to quickly switch the .NET Framework version that the aspx pages are compiled against.I am not sure why you would want to run both the versions of ASP.NET on the same box at the same time as 1.1 is backward compatible to 1.0, but I am sure there are many reasons which make total sense – I just cannot fanthom any at this time. Internally, it uses the command line tool called ASPNET_REGIIS that is included in the Microsoft.NET Framework.

Personalised Web Search (Google)

If you did not hear, then google recently launched its Personalised Web Search feature which is still not Beta (its on their lab page). Here you can go and setup you interests (essentially your scope) and then you can narrow down your results to only within the context of that scope. Quite interesting when you are looking for something which is also very generic or a completely different topic.

What the heck am I doing here?

Some of you out there have emailed me and expressed interest in knowing a little more on what I do and what is the purpose of this blog and what the heck does it run on? Now I am absolutely amazed that there are people out there reading this. My question to you is, what the heck are you doing reading this, don’t you have work or something to do? Anyways, what can I say, I am a Geek! W00T! Computers is something (probably the only things) I know a little about and have been playing with them ever since I can remember. Computers even today is first a passion, then a hobby and then a career, I am just very happy I am doing something I love doing and getting paid for it. 🙂

This came into being because many of my dear (and geeky) friends and colleagues bugged the hell out of me on where is my blog? They figured that I would be the first one out there with one. But with all the work and travel (due to work) I don’t get the time. Even now don’t expect an entry here everyday – it might now and then for the most part.

I run a modified version of .Text (very slightly modified) which is an excellent open source implementation called .Text, check them out on Gotdotnet. This ofcourse is running .NET with the backend being on SQL Server. .Text is pretty flexible and you can do a lot of stuff, I just have not had the time that is all. If you do go that way, try it out locally and let me know if there are any problems. Though Scott Water, the author of .Text has an excellent Wiki which I would ofcourse recommend to check out first. But as I said, if you want any help feel free to ping me, I cannot guarantee a reply immediately, but I will certainly get back. 

I wanted to find a good (and cheap) hosting company, so after looking high and low, I came across discountasp.net – I have been with them for 2-3 months now – not very long when compared to one of my other domains, where I have been for the last 6 years or so.

Abstraction and Efficiency (in c++)

Bill Venners has an interesting article on Abstraction and Efficient. He says that Abstraction is a mechanism to understand things. A good analogy he lays out is in Mathematics. Where expressing a solution means we really did understand the problem and not write up some code to try a “special case“. Also states that it is important to generalise and see one particular problem as a example of the class of problems. You can read the article here, Part I (c++ style sweet spot) can be found here and Part II (modern c++ style) here.

Microsoft plans laptop tune-up with Longhorn

In Longhorn MS is looking to add some laptop-specific features designed to make portable machines easy to use as consumer devices such as portable DVD players. They are also looking to add a separate UI that could be “instantly booted up“ to play movies, music, etc. – hence getting you the the experience more comparable to using consumer electronics devices.

For someone like me who is mostly on the road, this would be good, if I can cut down to one “device” (i.e. the laptop) along with the chargers, wires, etc that I carry – my shoulders would certainly thank me.