Tag Archives: node.js

npm Couldn’t read dependencies (or “So that’s why people don’t develop node apps on Windows”)

Recently, I decided to put the the you-R-here application on github. Since I normally work in Windows, I setup a quick development environment on my Windows 8 machine at home. I installed Visual Studio 2012 Express, Notepad++, and the node.js install package that comes with npm. While those things were installing I thought to my self, “Self, why don’t I know anyone who develops node.js apps on Windows?”. Self didn’t say anything, so I went ahead and copied the files needed over from my Mac.

I chose not to include my node_modules folder in github, and to make things easier, I created a package.json file that listed the dependencies of the project. Doing so would (should?) allow anyone setting up the project from scratch to just run “npm install” in the root project folder to pull down all of the dependencies.

To test this, I deleted npm_modules and ran ye old command in PowerShell and… AND…

npm Couldn’t read dependencies

…an error was displayed. As shown in the screenshot above, I was presented with an error message that read…

Couldn't read dependencies
Failed to parse json
Unexpected token <nonprintable character here>
File: <path>\package.json
package.json must be actual JSON, not just Javascript.

A heavy sigh followed the error message, but wasn’t displayed on screen. Did I get the format wrong – leave off an ending curly brace or the like? I headed over to jsonlint and pasted the code in. It verified fine.

running jsonlint on my package.json file

The only other I could think of that were different than “usual” scenarios were that a) I was using Windows and b) I had created the file in Visual Studio. I couldn’t do much about the first other than switching back to a different OS, so I decided to instead try and create the package.json file somewhere other than Visual Studio. I deleted the file, created it in PowerShell, added it back into my Visual Studio project, added the contents back, and viola! – the dependencies installed and the app ran fine.

Creating the package.json file in PowerShell

Installing the dependencies from package.json

Packages install fine when package.json is created in PowerShell

you-R-here running after installing dependencies from package.json

After some poking around, I looked at the encoding of text files created by both Visual Studio and PowerShell (Notepad++ makes it easy to find by highlighting the current encoding underneath the “encoding” menu). As it turns out, npm doesn’t like UTF-8 encoded files (the Visual Studio default), but it does seem to work fine with ANSI (the PowerShell default).

packages.json created from Visual Studio 2012

packages.json created from PowerShell

I couldn’t find anything in the npm documentation that stated that the the file needed to be ANSI-encoded, but the moral of the story is to make sure your package.json files are ANSI-encoded.

Tagged , , , , , ,

you-Я-here (or “Where Node Target Process Has Gone Before”) – Part 3

In our previous post, we looked at how to setup a Node.js development environment on a Mac. In this post, we’ll write some minimal code to get a functional web-site going, and get ready to style it up and push it out to someplace useful.


It seems many Node.js projects have a main file named “index.js”, so I chose that convention for mine as well. My index.js file is below. Read through the code first. We’ll discuss it in detail below.

var app = require("express").createServer();
var io = require("socket.io").listen(app);
var tp = require("./targetprocess");
app.get("/", function(req, res) {
res.sendfile(__dirname + "/index.html");
app.get("/index.css", function(req, res) {
res.sendfile(__dirname + "/index.css");
//users currently connected
var _usernames = {};
//Global TP options
username: "myusername",
password: "mypassword",
url: "https://mycompanyname.tpondemand.com/api/v1/&quot;
io.sockets.on("connection", function(socket){
//get completed user stories and bugs for the current iteraion
tp.api("getEntitiesForActiveIteration", function(entities) {
//console.log("returned from TP call");
socket.emit("entitiesretrieved", entities);
//when the client emits "adduser", this listens and executes
socket.on("adduser", function(username) {
//store username in the socket session for this client, and in global list
socket.username = username;
_usernames[username] = username;
//tell the client that he or she has been connected
socket.emit("updateaudit", "SERVER", "you have connected");
//tell everyone else that he or she has been connected
socket.broadcast.emit("updateaudit", "SERVER", username + " has connected");
//update the list of users on the client side
io.sockets.emit("updateusers", _usernames);
//user changes active item
socket.on("changeactiveitem", function(activeItemId, activeItemName) {
socket.emit("updateaudit", "SERVER", "you changed the active item to '" + activeItemName + "'");
socket.broadcast.emit("updateaudit", "SERVER", socket.username + " changed the active item to '" + activeItemName + "'");
io.sockets.emit("activeitemchanged", activeItemId);
//handle client disconnects
socket.on("disconnect", function() {
//remove the username from the global list
delete _usernames[socket.username];
//update the list of users on the client side
io.sockets.emit("updateusers", _usernames);
//tell everyone that the user left
socket.broadcast.emit("updateaudit", "SERVER", socket.username + " has disconnected.");

view raw


hosted with ❤ by GitHub

The first things to notice are the “var xxx = requre(“something”);” lines at the top. Node.js uses something called a module loader to load in modules (which in Node.js are just other *.js files). It uses this mechanism to decide what depends on what, in a fashion (to me, at least) that seems reminiscent of the AMD API in RequireJS or even the IoC pattern in type-safe languages.

For us, we require express and socket.io, which we talked about in the previous post. The third requirement, something called “targetprocess”, is just another js file that I wrote that encapsulates access to the Target Process REST API.

Second is a line that tells the express application variable to listen on port 8080. This is a good practice for dev macines where something else, such as a web server, might normally be listening on the usual port 80. After that are a couple of routing calls. These calls tell the routing functionality in express to route all requests for the root web site to a page called “index.html”, and all requests to the root index.css file to a similarly named file in the directory requested.

Next is a variable to hold the names of all users currently connected to the system, and a call to the targetprocess.js component to set the username/password combination for the Target Process (it uses basic authentication) and the URL to its API.

Then come the Socket.IO calls. Socket.IO contains libraries for both server and client-side functionality. Without going through every line in detail, here is a brief summary of the methods that I use…

  • socket.emit – sends a “message” (I would think of it as creating an “event” to which the client html page can respond) to the client represented by “socket”
  • io.sockets.emit – sends a message to all clients connected to the application (including the one represented by “socket”)
  • socket.broadcast.emit – exactly like io.sockets.emit, except that the client represented by “socket” does not receive the message. In other words, this sends a message to everyone except “yourself”.

Documentation is pretty limited on the main Socket.IO site, but check out the links provided in the answer here for more information.

Here in index.js, we are handling various messages. Again, without going line-by-line, below is a list of each with a rough description of purpose…

  • connection – This is a built-in message that triggers whenever a client connects. All further code goes in the handler for this message, and the “socket” argument is used to send messages back to that client.
  • adduser – Triggers when a client sends its username to the server, shortly after connecting
  • updateaudit – A message sent from the server to the client whenever the server has some interesting information that should be displayed in an “Audit” or “Recent Activity” area of the client web page
  • updateusers – A message sent from the server to the client when the user list has been modified
  • changeactiveitem – Triggers when a client clicks on a particular item (user story or bug) on the web page to indicate that a new item is currently being demonstrated.
  • activeitemchanged – A message sent from the server to all clients to indicate that the currently active item has changed
  • disconnect – Another built-in message that triggers when the client represented by “socket” disconnects


Perhaps the next most interesting page is the client page, index.html. This page contains client-side JavaScript to handle the messages being passed by the server, and the HTML to render to the browser. Let’s look at the code…

<script src="/socket.io/socket.io.js"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.7.2/jquery.min.js"></script>
<script src="http://cdnjs.cloudflare.com/ajax/libs/underscore.js/1.3.3/underscore-min.js"></script>
<link rel="stylesheet" type="text/css" href="index.css" />
var socket = io.connect("http://myservername:8080&quot;);
// on connection to server, ask for username
socket.on("connect", function() {
socket.emit("adduser", prompt("Enter username:"));
// update the audit log when appropriate
socket.on("updateaudit", function(username, data){
var auditEntry = "<li>[<%= timeStamp %>]<b><%= name %>:</b><%= info %></li>";
$("#audit > ul").prepend(_.template(auditEntry, {timeStamp: (new Date()).toString(), name: username, info: data}));
// update users list when needed
socket.on("updateusers", function(data){
var userList = "<% _.each(usernames, function(username) { %> <li><%= username %></li> <% }); %>";
$("#users > ul").html(_.template(userList, {usernames: data}));
socket.on("entitiesretrieved", function(queryResults) {
var entitiesList = "<% _.each(items, function(item) { %> <li id='<%= item.Id %>'><div class='<%= item.EntityType.Name %>-icon'></div><%= item.Name %></li> <% }); %>";
var html = _.template(entitiesList, { items : queryResults.Items });
$("#items > ul").html(html);
$("#items > ul > li").on("click", function(event) {
socket.emit("changeactiveitem", this.id, this.innerText); //highlight the currently selected item when clicked
socket.on("activeitemchanged", function(itemId) {
$("#items > ul > li.highlight").removeClass("highlight");
$("#items > ul > li[id='" + itemId + "']").addClass("highlight");
<h2>Your Friendly Agile Demo Helper<h2>
<section id="users">
<section id="items">
<section id="audit">

view raw


hosted with ❤ by GitHub

You’ll notice several external script references at the top of the page. The first is Socket.IO, required to handle the client-side messaging functionality. After that are jQuery and Underscore, which I am currently only using for its JavaScript templating functionality. Both are loaded from content delivery networks to try and minimize the number of files I have to host, and to speed up delivery of the libraries.

Next we see the initial connection being made to the root of the web site, followed by a bunch of message handlers. Here is a rough list of descriptions of each…

  • connect – Built-in message for initial connection. The handler here sends back the “adduser” message along with the user’s name
  • updateaudit – Listens for audit messages from the server, and lists them out on the web page, in reverse chronological order
  • updateusers – Listens for messages from the server indicating that the user list has changed, and re-renders it to the web page
  • entitiesretrieved – Listens for the message from the server with teh same name, and renders the list to the web page. Also wires up a handler that will send a message to the server to change the active item in the list whenever the user clicks on one in the web page.
  • activeitemchanged – Responds to the message from the server by adding a highlight to the currently active item, and removing the highlight from all other items.

Oh, the Humanity!: An “Issue” With Underscore

One thing to note here is that as I was working with Underscore, I kept getting an error related to an equals sign being invalid. After much fighting, it turns out that whenever you reference a variable or object property inside your template, you ABSOLUTELY MUST put a space between the equals sign and the variable or object property.

i.e. “<%= name %>” works, “<% =name %>” does not


The targetprocess.js file, as discussed above, is simply a wrapper for calling into the Target Process REST API. Let’s look at the code…

var rest = require("restler"); //https://github.com/danwrong/restler
//var moment = require("moment"); //http://momentjs.com
var self = this;
var methods = {
globalOptions : {},
init : function(options) {
self.globalOptions = options;
self.globalOptions.format = self.globalOptions.format || "json";
getEntitiesForActiveIteration : function(callback, options) {
//todo: extend globalOptions with options to create localOptions variable
options = options || {};
var getOptions = {
username: options.username || self.globalOptions.username,
password: options.password || self.globalOptions.password,
parser: rest.parsers.json
var url = [];
url.push(options.url || self.globalOptions.url);
url.push(options.format || self.globalOptions.format);
url.push(encodeURIComponent("(EntityType.Name in ('UserStory','Bug'))"));
url.push(encodeURIComponent(" and (Iteration.EndDate eq '"));
//url.push(moment().format("YYYY-MM-DD")); //todo: use moment.js to make this work for any day (finished in current iteration)
url.push(encodeURIComponent(" and (EntityState.Name in ('To Verify','Done','Fixed','Closed'))&take=1&include=[Id,Name,EntityType]")); //todo: make this end states configurable
url = url.join("");
console.log("url is: " + url);
rest.get(url, getOptions).on("complete", function(result) {
if (result instanceof Error) {
console.log("ERROR: " + result.message);
} else {
api = function(methodName, callback, options) {
console.log("Calling " + methodName);
if (methods[methodName]) {
return methods[methodName].apply(this, Array.prototype.splice.call(arguments, 1));
} else if (typeof methodName === "object" || !methodName) {
return methods.init.apply(this, arguments);
} else {
throw new Error("Method " + method + " does not exist in tp.api");
exports.api = api;

The module requires the restler module which, as discussed in previous blog posts, provides a simple way to interact with REST-based APIs, not unlike the httparty Ruby gem.

In this module, I used a convention often used in jQuery plugins. The module has but one method, called “api”. If the user uses this method in the “normal” way, it takes a string argument indicating the name of the action to perform, followed by a JavaScript callback function that will be called when the requested information is returned from Target Process. If, on the other hand, the user wants to set global options, he or she can call the method with only one object as an argument, and the username, password, and url will be extracted from that object. This convention can be found in the “api” method toward the bottom of the code file.

Other than that, the only action method available is “getEntitiesForActiveIteration”. This method retrieves all UserStory and Bug objects from Target Process that were completed in an iteration ending on a specific date. Right now, this specific date is hard-coded, but I hope to get that calculated or passed in later, which should be easy enough. For more details on how to form these HTTPGet queries to retrieve Target Process objects, see the Target Process REST API.

One final thing to note is the line at the very bottom of the code file. Whenever a Node.js module is written, you must declare which methods, if any, will be exposed publicly to the modules that use your module. In our case, only the “api” method is exposed publicly.

Oh, the Humanity!: An “Issue” With The Target Process REST API

On my first few attempts, I kept getting a message back from Restler saying that it couldn’t parse the JSON coming back from Target Process because the less-than sign (<) was an invalid character. At first, I thought that this was caused by the fact that we sometimes put HTML into the comments or descriptions of our Bug and UserStory objects. But I narrowed the query to only select one item, and the issue persisted. As it turns out, I had a bad query that I hadn’t tested in the browser, and Target Process was giving an error message back in the form of an HTML response. Apparently when errors occur, they appear in the form of HTML-formatted responses, even if you ask for JSON in the request URL.


Last (and probably least) is index.css. As of now I have only some minimal formatting, just to make sure that the highlight of the active item shows up, and that my lists aren’t blindingly ugly. Code is as follows…

font-family: 'Segoe UI', Tahoma, Arial, Helvetica, Sans-Serif;
margin: 12px;
list-style-type: none;
margin: 0;
padding: 0;
background-color: yellow;

view raw


hosted with ❤ by GitHub

Testing the App

To test the app, fire up bash (Terminal Window on a Mac), navigate to the path where index.js exists, and type “node index.js”.

Starting the web site

Then fire up a couple of instances of your favorite web browser (they need not be on the same computer, just able to see the web site – I used my Windows computer), type your username on each…

…and then start clicking on items in the list.

Clicking on an item in one browser will make that item highlight in every browser that is connected.

Next Time

Next time, we’ll add some styling to improve the look of the application, and (hopefully) get it running in a real website somewhere (like hieroku).

Tagged , , , , , , , ,

you-Я-here (or “Where Node Target Process Has Gone Before”) – Part 2

Last time, we looked at the idea of having an agile iteration demo helper written in Node.js. In this post, we’ll be looking at how to setup the Node.js environment, and get ready to write some code.

Installing Node.js

Node.js originally wouldn’t run on Windows, at least not natively anyway. You had to install Cygwin and recompile Node.js to get it to run. These days, the windows platform is a first-class citizen for running Node.js. Heck, you can even get Node.js running on IIS. For these posts, however, I chose to setup the environment on a Mac Mini.

Installation itself was a breeze. I just went out to the Node.js site, clicked the big “download” button, and the went through the package installation wizard as shown in the screenshots below…

Node.js site

After clicking big “download” button

Package downloaded

Package installation wizard

One thing to note here is that in addition to Node.js itself, the installation installs the npm, which is an abbreviation for “Node Package Manager”. The npm lets you install useful pieces of code into your project from the web. If you have a Microsoft background, think NuGet packages.

A Place For Code

Before we start installing packages, we’ll need a place to put our code. I just created a folder at “Documents\dev\nodejs\youRhere” for this project (I wasn’t cool enough to make the “R” backwards in the folder name)…

My code folder

Package Installation

Okay, now back to packages. Which packages will we need, exactly?  First off, we’ll need two pretty popular ones – Socket.IO, and Express.

Socket.IO is a library that makes is super easy to do real-time communication in web applications. It is compatible with a variety of browsers and communication technology, and will select the “best” technology and gracefully fall back to “less optimal” technologies when needed. It defaults to using HTML5 Web Sockets, for example, but will fall back to using Adobe Flash Sockets if your browser is not HTML5-compliant. Don’t have Flash installed? Then it will fall back again to use AJAX long polling techniques. This pattern continues until it runs out of options and deems your browser too ancient to use, or finds a compatible technology. To install Socket.io, just fire up ye ol’ bash shell (called “Terminal” in OSX in Finder->Applications->Utilities), navigate to the code folder we created in the previous section, and install the package using the syntax “npm install <pkg>”. “But I don’t know bash commands, and I don’t know how to do a Google search to find them!”, you say? Never fear. Just check out the screenshots below for more info…

Running Terminal from a Finder window

Terminal (Bash shell)

Navigate to code folder

Install the package

Express is the second package we’ll need. The Express web framework makes it easy to do routing, render views on the client, and generally eases the process of making Node.js-based web applications. To install it, just run “npm install express” in the same Bash window.

Install Express

One other package we know we’ll need to install is called “restler”. Restler makes it easy to work with REST-based APIs like the one used by Target Process. Installing it is similar to the others…

Install Restler

Next Time

Now that we have the environment and all (hopefully) of the packages that we’ll be needing, next time we’ll be able to start coding the application.

Tagged , , , ,

you-Я-here (or “Where Node Target Process Has Gone Before”) – Part 1

One of the things I’ve been meaning to take a look at for a while is Node.js. I’ve always had an interest in JavaScript, and both loved and hated it in years past. It would seem only natural that a super-fast, server-side flavor of the language based on Google’s JavaScript runtime might be interesting as well.

To do that, I need a self-assigned project. To make that project more real-world, inspiring, something about which I could be passionate, or what have you, I searched around for a bit to find a Node.js project that seemed like something that might actually be used (or at least have the potential to be used) by real people. To prevent the related blog posts from taking a year, I also searched around for something small enough that I could crank out in a few nights of work at home.

The Idea

After mentioning it to @ryoe_ok, he had a great idea to enhance our agile iteration demo experience for remote attendees. The basic premise was something like this: At our company, we currently use GoToMeeting for agile iteration demos. The product owner and other stakeholders, scrum master, and development team all attend the demos. Sometimes, however, the current feature being demonstrated isn’t as clear as it could be. In order to help with that, I could create a web page. People attending the demo would open the web page in addition to GoToMeeting. The page would simply list the user story currently being demonstrated, and the name of the developer currently demonstrating it. After the demo of that story is finished, the story (and perhaps the name and picture of the demonstrator) would change in real-time, making it very clear for anyone viewing the page exactly what is going on at the moment. He called it “You are here.”

The asynchronous nature of Node.js and its support for web sockets seemed a great fit for the idea. Shortly after finishing the discussion with @ryoe_ok, I promptly stole borrowed his idea, and then elaborated on it a bit. First (just for fun), I thought about the name. I changed it to the more hipster-esque “you-Я-here”. Second, I actually thought of something useful. We are currently using Target Process to help us do agile project management in our semi-distributed team. It has a pretty extensive API for querying user stories, bugs, and other objects. If I could somehow use Node.js to communicate with that API, no one would need to type in the user stories that needed to be demoed. The web page could just figure that out based upon the current date. Just think of all those poor keystrokes we could save!

The UX

Now that the general idea is in place, we need to take a few minutes to flesh out some semblance of a design for the user experience. As far as wireframe tools go, my first choice would probably be Balsamiq, but since it costs around $80 at the time of this writing, and I’m too cheap to buy it  I’m under austerity policies that prevent me from buying it  I’m trying to be frugal, I’ll use Lumzy instead, which is free forever (as far as you know). Here is what I have as a first-pass…

First draft of user interface design

That probably won’t win any design awards, but it shows the general idea for now. We can come back later and make things better if we need. Perhaps, for example, some visual cue could be given such as a fade in/fade out when the data changes.

Next Steps

Now that we know what we want to do, we need to install Node.js and setup the development environment. We’ll be looking at that in the next post.

Tagged , , ,