Detecting Global Pollution with the JScript RuntimeObject Sunday, 11 April 2010

This article is about debugging with JScript's RuntimeObject (msdn). All of the examples work in IE 5.5+, though most do not work in any other browser.

Leaked Global Identifiers

Say you accidentally created a global property, as in the following:

function playRugby(players) {
  var items,
      i;
      len = items.length; // Global.
}

function kick() {
  var x = 10
      y = 11; // ASI makes y global.
}
When playRugby is called, a global property len is created, if it does not already exist, and then assigned the value of items.length. Likewise, when kick is called, a global property y is created.

These globals are unintentional. They break encapsulation and leak implementation details. This can result in conflict and awkward dependency issues.

To detect accidentally created global identifiers, we can loop over the global object using for in. Firebug provides this convenient global inspection under the "DOM" tab.

Everybody's Favorite Browser

Unfortunately, in IE, the for in won't enumerate any global variables or function declarations, as seen in the example below.

Example Enumerating the Global Object

// Property of global variable object.
var EX1_GLOBAL_VARIABLE = 10;

// Property of global object.
this.EX1_GLOBAL_PROPERTY = 11;

// Property of global variable object.
function EX1_GLOBAL_FUNCTION(){}

(function(){
  var results = [];
  for(var p in this) {
    results.push(p);
  }
  alert("Leaked:\n" + results.join("\n"));
})();

The result in IE contains a mix of window properties and one the four user-defined properties: EX1_GLOBAL_PROPERTY.

So what happened to the other three user-defined properties? Why didn't they show up in the for in loop?

It turns out that enumerating over the global object will enumerate properties assigned to the global object and will not enumerate global variables.

An educated guess as to why global properties are enumerated but global variables are not might be that JScript gives global variables (declared with var), the DontEnum flag. Since the global object is specified as being the global Variable object, this seems like a likely explanation. It would be nonstandard, but it would explain the behavior in IE. Eric Lippert, however, provided a different explanation: The global object and the global variable object are two different objects in IE.

According to MS-ES3:

JScript 5.x variable instantiations creates properties of the global object that have the DontEnum attribute.

Enumeration Solution: The JScript RuntimeObject

To enumerate over global properties, use the JScript RuntimeObject method. Instead of enumerating over the global object, as you would use in a normal implementation, enumerate over an object returned by the global RuntimeObject method.

var GLOBAL_VAR1, 
    GLOBAL_VAR2, 
    GLOBAL_VAR3 = 1; 
    GLOBAL_PROP1 = 12;

function GLOBAL_FUNCTION(){}

if(this.RuntimeObject){
    void function() {
        var ro = RuntimeObject(),
            results = [],
            prop;
        for(prop in ro) {
            results.push(prop);
        }
        alert("leaked:\n" + results.join("\n"));
    }();
}
IE Result

The result in IE 8 and below includes (among other things, including window) GLOBAL_FUNCTION, GLOBAL_VAR3, and GLOBAL_PROP1, in that order, as they were evaluated in. Notice that neither GLOBAL_VAR1 nor GLOBAL_VAR2 were included. It appears that RuntimeObject does not accumulate any variables that were unassigned to. According to Microsoft's documentation, this is not the specified behavior (more on this below).

Microsoft RuntimeObject Documentation

The JScript RuntimeObject is a built-in extension to JScript. JScript defines seven additional built-in global methods: ScriptEngine, ScriptEngineBuildVersion, ScriptEngineMajorVersion, ScriptEngineMinorVersion, CollectGarbage, RuntimeObject, and GetObject. These objects are all native JScript objects, not to be confused with host objects.

For RuntimeObject, Microsoft JScript Extensions [MS-ES3EX] states:

The RuntimeObject function is used to search a global object for properties with names that match a specified pattern. The function only locates properties of the global object that were explicitly created by VariableStatement or FunctionDeclaration functions, or that were implicitly created by appearing as an identifier on the left side of an assignment operator. The function does not locate properties that were created by means of explicit property access on the global object.

Superficial testing indicates that Microsoft's documentation is wrong.

The returned object does not includes all identifiers that were added to the Variable object; only those identifiers that have been assigned a value. Whether or not they were created from VariableDeclaration, FunctionDeclaration, or assignment as global properties does not matter.

Example of Finding Identifiers Created By FunctionBindingList

All identifiers in a FunctionBindingList of a JScriptFunction will become properties of the containing Variable object, so, for example:

var foo = {}, undef, ro;
(function(){ function foo.bar, baz(){} })();
ro = RuntimeObject();
alert([ro.foo.bar, "undef" in ro].join("\n"));
IE elerts
function foo.bar(){}
false

Browsers other than IE running JScript can be expected to throw SyntaxError upon parsing the FunctionBindingList of JScriptFunction production. This is to be expected, as it is a syntax extension.

Bookmarklet

As a bookmarklet:
javascript:(function() {var ro=RuntimeObject(),r=[],i=0,p;for(p in ro){r[i++]=p;}alert('leaked:\n'+r.join('\n'));})();
JScript Syntax Extension

The earlier example "Finding Identifiers Created By FunctionBindingList" mentioned the JScript Extension JScriptFunction. In case the name is not a dead giveaway, this is a JScript language extension. The production for JScriptFunction is:

JScriptFunction : 
function FunctionBindingList ( FormalParameterListopt ) { FunctionBody }
RuntimeObject(filterString): The filterString Parameter

The RuntimeObject method accepts an optional filter string to match identifiers. Unfortunately, filterString is not converted to a regular expression but is used for substring matching with optional leftWild and rightWild, defaulting to *.

This means that, for example: filterString = "a*" would match identifiers a and a1 but not ba.

Conclusion

Documentation bugs and shortcomings aside, the RuntimeObject provides a useful alternative to the problem of enumerating global properties in JScript. An advantage with RuntimeObject is that it only includes user-defined properties, with the exception of the global window property.

The aforementioned bookmarklet provides a convenient way to check a page to see the globals that have been accidentally created (it also shows that this site is not a shining example of keeping the global object clean).

Other Applications for RuntimeObject

Cross Browser Identifier Leak Bookmarklet

Writing a cross-browser identifier leak detector is the next logical step to an IE-only identifier leak detector.

Automated Identifier Leak Detection

Checking for accidental global identifiers should be automated.

The YUI Test unit test framework provides hooks for TEST_CASE_BEGIN_EVENT and TEST_CASE_COMPLETE_EVENT . These events can be used to inspect the RuntimeObject and catch global identifier leaks that occur througout the runtime execution of program code.

In TEST_CASE_BEGIN_EVENT, inspect the RuntimeObject and save the result. In TEST_CASE_COMPLETE_EVENT, inspect the RuntimeObject again and compare the results with results saved during TEST_CASE_BEGIN_EVENT. Next, for each property that appeared in TEST_CASE_COMPLETE_EVENT but was not present in the result saved from TEST_CASE_BEGIN_EVENT , a global identifier has been leaked and a test case warning can be logged.

References

  • [MS-ES3EX]: Microsoft JScript Extensions to the ECMAScript Language Specification Third Edition.
Posted by default at 4:23 PM in Browsers

Ebay Facilitates Fraud Thursday, 18 March 2010

Won an auction of eBay for:

"You Won eBay Item:DELL E1705 INTEL DUO 2 GHZ, WINDOWS VISTA ULTIMATE (290295125189)"

I got that laptop nearly two weeks after I paid for it.

Shortly thereafter, I got it in to a local tech who, after about four days, informed me that the copy of windows that was installed was unlicensed.

It turns out MS Office, which was also advertised as being included in the auction, is also pirated.

Paypal Claim

I immediately filed a complaint with the payment system, Paypal "Item significantly not as described". The claim was requesting the seller to provide a license key for Windows.

The seller could not provide a license key for windows because he does not have one. He decided to make the irrelevant excuse "I lost the CD" to paypal, and somehow, that worked.

On March 20, 2009, I receieved the email from paypal stating:

We will notify you if further action is required.

I called paypal and the representative said I would receive contact from paypal by April 17 (IIRC), however that never happened and Payapl closed the claim on May 24, 2009, notifying me with the following:

Paypal Case Details

We have concluded our investigation into this case. Unfortunately, 
at this time we are unable to decide this claim in your favor.

-----------------------------------
Case Details
-----------------------------------


As you can see, there are no "case details" there; the entire section is completely blank.

As soon as I received this, I called Paypal to ask why the case had been closed. The Paypal representative could not provide a reason, nor could he state what, if anything, paypal had done. He reopened the case and stated that a Paypal representative would contact me.

Paypal did not contact me, but instead closed the case again.

I went through the process of calling Paypal again, waiting on hold for a long time, and again talking to a rep. The case was reopened, and then once again re-closed, without any evidence that paypal had actually done anything to investigate the claim.

I have saved all of the email communications with Paypal. The responses from Paypal indicate that the paypal representatives failed to read my messages and failed to read the auction title which was included in the message.

Paypal has provided no evidence to having done anything to investigate my claim.

Back to Ebay

After Paypal failed, I called ebay. I made many calls to ebay, each time having to restate everything from the beginning. I received follow-up email from rswebhelp@ebay.com stating that I should contact the police, file a complaint with IC3, and file a mail fraud complaint with USPS.

Ebay Action

Like paypal, Ebay ignored many, if not most of my emails. Of the emails that I received a reply to, the responses do not include answers to the questions. They are a top-reply of mostly irrelevant parroting of what appears to be copy'n'pasted information on how to call the police, file a claim with the post office, or contact IC3. I have done all of those things.

Ebay stated that they work closely with the police. I provided rswebhelp with the police report number and requested for them to call the police but the request was ignored. What ebay says and what ebay does do not match.

The San Francisco Police officer I reported to told me that the case would be only paperwork for them. He would not even look at the URL. I cannot force them to change how they operate.

The Seller

I followed up with Dave Kaercher by sending email and by calling. The phone message I left was not replied to and the email (all emails I have sent him) was ignored. I did get through a week later and told Dave the problem and that I was seeking a refund of money I paid. Dave said "we don't have a deal" and that was the end of the conversation.

The computer has not provided me with the use that could be expected out of a computer with legal, licensed versions of Windows Office.

Instead of using Windows, I have been hobbling with a cracked OS for 1 year. The OS frequently restarts (in failed attempts to run important security updates).

Instead of the expected use of Microsoft Office, Launching Office errs with: "

Microsoft Office Genuine Advantage

  [logo] This copy of Microsoft Office is not genuine.
  Please excuse this interruption. This copy of Office did not 
  pass validation. Click Learn More for online details and help 
  identifying the best way to get genuine Microsoft Office.
              [Learn More] [Remind Me Later]

Clicking "Learn More" leads the web page: Genuine Microsoft Software

I have actively pursued this for over a year, with several emails and calls to ebay, paypal, and Dave Kaercher. I am posting the seller's personal information on my site.

Dave Kaercher

Dave Kaercher: I told you I would do this. I clearly requested licensed copies of what I paid for. I stated this in emails to you and in the paypal claim. You made excuses and ignored those.

I told you two weeks ago by email (which you ignored) and by phone call two weeks ago that I would post your information on my site.

Dave Kaercher's Personal Info

Dave Kaercher GRI, QSC User ID: redbirds04 Name: Dave Kaercher website: wesellmore.com City: Colorado Springs State: CO Country: United States Phone: (719) 282-1681

Lesson Learned

Don't buy things off eBay!

EBay knowingly facilitates fraud. EBay makes money by helping criminals defraud consumers and as such, is guilty of fraud.

Dave Kaercher is in good standing with ebay, probably defrauding other victims. He continues to promote auctions for various things including fake steroids over the past year. It is clear that eBay does not care to take this seriously.

Anyone shopping for items should consider that what is sold on ebay might be illegitimate (pirated, counterfeit, etc). In the case that the item is illegitimate, ebay probably won't do anything about it.

Technorati Tags:

Posted by default at 11:00 AM in Uncategorized

Myopia and the Opera 10 User Agent String Friday, 29 May 2009

Opera has conceived a silly tactic planned for Opera 10 user-agent string.

The problem is that there are scripts that expect the browser major version to single-digit and will fail if it is not.

Since "10" not a single digit, these scripts fail.

Opera has mitigated that problem by changing the user-agent to 9.80 and publishing the following warning:

Browser sniffing ? unless you?re writing a web stats application ? is always a bad idea. It?s a misguided attempt to send different content to different user agents. This is never scalable ? you can?t change every website you?ve ever made every time a new browser version comes out. It is also not future-proof, as highlighted by this article.

Ineffectual and meaningless little blurb there. Those badly written sites that used (poor) browser detection will not break from Opera. Opera spoofing their own user-agent string helps reaffirm the misconception that the authors' browser detection worked. Posting up a little warning that not everyone will read does not make an example.

The blurb states that Browser detection is used "to send different content to different user agents". Not always true. In fact, browser detection is more often used on the client to work around an perceived incompatibility. Since Opera is wrong on that count, it makes the blurb seem even less relevant, as an author who read it might still try to justify or rationalize his approach by saying "but that's not why I used browser detection."

Browser detection scripts cause forwards-compatibility and maintenance problems. However, to not be able to parse out a number is not only not smart, it shows very poor coding skill.

Opera states that version 11 will have "11" in the user-agent string.

My opinion is somewhat in line with Doug's on this one (that Yahoo 360 URL is an awful URL).

If you are a developer, check your code. It really isn't hard to do this stuff correctly. It really isn't.

Where I disagree with Doug is "Opera has been forced to lie."

Opera developers made a decision to lie, as explained by Opera. They were not forced.

An alternative to that choice is for Opera to not cater to badly authored pages and simply let them break.

Breaking sites is bad in the short term because it renders pages unusable. However, it is good in the larger scheme of the web in the long run. By driving home a hard lesson, Opera could teach developers to not use browser detection by providing an historical lesson.

The first sensible opinion on the matter was Hallvord's post from December, 2008., where he pointed out that Bank of America and Live.com failed in Opera 10. The entry describes the reason: Faulty parsing of the User-Agent string, and redirecting to the "not supported page".

You'd think that with the intense development Microsoft has been lavishing on live.com they would have found somebody capable of writing a usable browser sniffer (or ideally a person clever enough to say "wait, we don't really need one - what if we just use feature detection instead?"). Think again..

Of course, Microsoft has been advocating detection "best practices" for years, despite well reasoned arguments to stop doing that (G. Talbot, T. Zijdel).

Opera should be less myopic and stop worrying about breaking badly authored sites. Web developers should be less myopic, and build maintainable, forwards-compatible solutions.

Posted by default at 12:06 AM in Browsers

Function.prototype.bind Thursday, 11 September 2008

Function Binding

A bind function wraps a function in a closure, storing a reference to the context argument in the containing scope.

This allows the bound function to run with a predetermined context.

Variable this

When a function is passed as a reference, it loses its base object. When the unbound function is called, the this value is the global object.

How Binding Works

By storing a reference to the desired object in a closure, this argument can be bound.

In it's simplest form, bind looks like:-

Function.prototype.bind = function(context) {
  var fun = this;
  return function(){
    return fun.apply(context, arguments);
  };
};

Why Binding is Useful

Binding is often necessary when passing function references. For example:-

var updater = {
  fetch : function() {
    alert(this.time++);
  },
  time : 0
};
// setTimeout(updater.fetch, 500);
setTimeout(updater.fetch.bind(updater), 500);

The commented-out call to setTimeout would result in a call to updater.fetch with the global object for the this argument. this.time would be undefined, and this.time++ would result in NaN.

A bind function that does only binding accomplishes a trivial task. In most cases, a closure can just be used where binding is needed.

Binding in the Wild

Most JavaScript libraries handle binding internally. These libraries also include a partial apply for their bind function.

Partial Apply

Partial application is setting parameter values of a function call before it is called. A partial apply function usually looks like:-

/** 
 * Return a function that prepends the 
 * arguments to partial to this call and 
 * appends any additional arguments.
 */
Function.prototype.partial = function() {
  var fun = this,
      preArgs = Array.prototype.slice.call(arguments);
  return function() {
    fun.apply(null, preArgs.concat.apply(preArgs, arguments));
  };
};

This allows us to program in a dynamic, functional, less OO way. For example:

function setStyle(style, prop, value) {
  return style[prop] = value;
}

// Create a setBgColor function from 
// partial application of setStyle.
var setBgColor = setStyle.partial(document.body.style, "background");

// Change the body's background color.
setBgColor("red");
setBgColor("#0f0");

Disadvantage

Partial application requires an extra function call, plus a call to concat for the extra arguments.

Partial application can make debugging trickier, since there is an extra layer of indirection to the real method.

Bind + partial apply can be used to force the this argument of a prototype method to always be the of the instance. This is inefficient and often leads to messy, tangled function decomposition. Libraries that bind every method do so out of ignorance of the language, and are best avoided.

EcmaScript New Language Feature

The forthcoming version of EcmaScript (now called EcmaScript Harmony) will include Function.prototype.bind(context). A native bind should outperform any other bind function.

This was something Peter brought up for EcmaScript 4, but appears to be making way into the revised EcmaScript Harmony.

Mark Miller wrote out a "self-hosted" version of EcmaScript's proposed Function.prototype.bind. It is:-

Function.prototype.bind = function(self, var_args) {
   var thisFunc = this;
   var leftArgs = Array.slice(arguments, 1);
  return function(var_args) {
    var args = leftArgs.concat(Array.slice(arguments, 0));
    return thisFunc.apply(self, args);
  };
};  

After looking at the ES Harmony proposal, and looking at a few versions of bind functions, I decided to write a better one that does exactly what the ES Harmony's bind does, but with greater efficiency than the current libraries offer, and whose, length property was 1.

Although unnecessary, this is a welcome addition to the language. A native bind will outperform any user-defined bind function and will result in fewer closures.

The Rundown

Before I give a critique and rundown, I have a test.

Library Comparison Test

  1. Garrett's Bind

    This bind was, by far, the most efficient in tests #1, and #4, and nearly ties Base 2 in test #2 (Base 2 was about .5 ms faster) . This function requires no additional code or functions.

  2. Base2 bind

    Second performance-wise.

    Requires only a top level _slice function (trivial), and performs a strategy for extra arguments.

  3. Dojo's hitch

    Dojo was fast with pure bind, but slower with partial apply.

    This function requires many other functions and has an additional complication of accepting strings and arguments of different order.

  4. Ext-js Function.prototype.createDelegate

    Performance was slow. This function requires no importing of external functions.

  5. Mark Miller's bind

    Requires no external dependencies. While it gets a 10 for simplicity and aesthetics, this function was not as fast, and for pure bind (no partial apply) was not nearly as fast as it should be.

  6. Prototype's Function.prototype.bind

    Performance was fair. Requires several extra functions + browser detection. The function is used very heavily internally.

  7. Mootools Function.prototype.bind

    Performance times were poor and the results for two tests were wrong. were wrong.

    The entire mootools.js is required for the bind function. The library adds a $family property, and makes other changes to Array.prototype.

  8. YUI 3 bind

    Performance time was fair. The results for test#2 are wrong because the call's arguments are prepended, not appended. This is by design.

    Having the call's arguments prepended is an unusual design decision. It might seem unintuitive, and confuse developers who are used to the more common version, as seen in Prototype, Base2, and the official EcmaScript proposal.

    The amount of code YUI's bind depends on is staggering.

Pass the Parmeźan

Many of the libraries have long chains of function calls. A bind function does not need and should not require the inclusion of several other functions.

Prototype JS requires the $A function, which requires Prototype.Browser to determine which $A function. Browser detection has absolutely no place in Function binding. ($A also calls toArray conditionally, but that will not happen in this case.)

Dojo is almost as bad. Dojo's hitch function has the arguments in reverse order and requires dojo.global, dojo._hitchArgs, dojo._toArray, and dojo.isString.

Mootools has very strewn code. The broken bind function required an additional 108 lines of Mootools.

YUI is the most grandiose. Function YUI.bind(f, o) is found in "oop.js". File oop.js requires over 120k of "prerequisite" files in yui.js and yui-base.js, coming to a total of over 160k. Just for bind. YUI 3 seems to suffer from over-engineering and BUFD, which is typical in waterfall shops.

YUI's 'array' module did not seem to load or evaluate properly, so code from the yui-base file was copy-pasted.

Worth Using?

The best bind functions are fast, do not require other library functions, and are fairly simple.

But is a Bind Function Necessary?

No. Binding can be achieved with an inline closure where it is needed and partial application is not necessary.

Here is example 1, without using bind.

var updater = {
  fetch : function() {
    alert(this.time++);
  },
  time : 0
};
setTimeout(function() { updater.fetch(); }, 500);

A native Function.prototype.bind will allow for cleaner binding, without the need for creating a closure. As native code, it will be faster and more reliable. Function.prototype.bind is not necessary, but is a welcome addition to the language.

Why All the Fuss?

Being aware of what libraries do and identifying and learning from the mistakes of libraries helps developers avoid such mistakes by learning what the library does. Developers do not need the burden of large, tangled, and often buggy library dependencies.

Look at The Code

What the library's bind function does and how the library uses that function internally is a step to take in assessing the library's quality.

A developer can make a more responsible and professional choice by avoiding a library that makes heavy use of a slow-spaghetti bind function.

My Version

The following function is the fastest bind function for pure bind (no partial apply). It is more than three times as fast as Base2, the second fastest bind function tested here.

Here is my version of the bind function.

/**
 * @param {Object} context the 'this' value to be used.
 * @param {arguments} [1..n] optional arguments that are
 * prepended to returned function's call.
 * @return {Function} a function that applies the original
 * function with 'context' as the thisArg.
 */
Function.prototype.bind = function(context){
  var fn = this, 
      ap, concat, args,
      isPartial = arguments.length > 1;
  // Strategy 1: just bind, not a partialApply
  if(!isPartial) {
    return function() {
        if(arguments.length !== 0) {
          return fn.apply(context, arguments);
        } else {
          return fn.call(context); // faster in Firefox.
        }
      };
    } else {
    // Strategy 2: partialApply
    ap = Array.prototype,
    args = ap.slice.call(arguments, 1);
    concat = ap.concat;
    return function() {
      return fn.apply(context, 
        arguments.length === 0 ? args : 
        concat.apply(args, arguments));
    };
  }
};

This function was not included in APE because it was not needed. This function may be used by libraries who wish to continue using a bind function with the benefit of faster performance.

Technorati Tags:

Posted by default at 5:20 PM in JavaScript

Find Element Position Friday, 4 July 2008

Most libraries try provide some functionality for finding an element's position, but fail in all but the simplest cases.

I created a test page to demonstrate the libraries' results of finding an element's position and the time it takes for them to succeed or fail.

Test Page

Compare JavaScript Libraries

Demos are Not Unit Tests

These two demos don't cover all of the complexities in finding an element's position. I didn't even touch tables. There are also complications with padding and border on HTML root element. Unfortunately, IE8b1 doesn't support CSS on the HTML root element (connect id 354453), so it can't be tested.

APE guards against all of the other problems by having a test suite that tests this function with combinations of these cases.

Technorati Tags:

Posted by default at 2:36 AM in JavaScript

 

*AnimTree
*Tabs
*GlideMenus
*DragLib