Breezejs navigation properties

The more I use BreezeJS the more I like it. I’m not using it with a .NET infrastructure so I’m hand rolling my metadata. Here’s a note of my findings for future reference.

key: The primary key for the entity. Can be a compound key. Required when new entities are created, loading by key and for reducing the amount of information required to describe navigation properties.

The science of relating entities together. We can have 1:1, 1:many and many:many (via a mapper collection) relationships. We can have parent to child relationships and inverse relationships (child to parent).

name: Arbitrary name of the property. In knockout land this will be the observable property that contains the related entity/entities

entityTypeName: The namespaced entity name of the related entity, eg Location:#Tinder

isScalar: Single(true) or many(false). For non scalar the related entities are embedded in an array, for scalar types the entity is embedded directly.

associationName: Arbitrary name of the relationship. Convention is <root entity>_<child entity>, eg Centre_Location. Must be unique, as it is used when describing relationships that have both forward and inverse relationships. In these cases the inverse relationship associationName must match the normal (forward?) relationship.

foreignKeyNames: Array of key names. These define the property on the root entity to use when fetching the related entity. This property value will relate to the primary key of the related entity. We do not need to specify the details of the related entity primary key here as this is defined in the related entity metadata.

Not sure how multiple keys would work but guess the ordering would match the compound key ordering in the related entity.

invForeignKeyNames: Array of key names. These define the relationship the other way around. These define the property on the related entity that matches the primary key of the root entity.

For most cases we only want to define normal or forward relationships where we state the parent to child relationship. For these cases the use of foreign and inverse keys depends if the relationship is 1:1 or many.

For 1:1 isScalar = true and the foreignKeyNames field is used.

For many isScalar = false and the invForeignKeyNames field is used.

I’ve not tried defining both foreignKeyNames and invForeignKeyNames for a relationship, upto now it’s only neccessary to define one. A possible exception might be for a many relationship. If the related entity does not map to the primary key of the root entity maybe it would be described with foreignKeyNames.

Using invForeignKeyNames can seem confusing, as it’s describing a forward relationship. It is useful though as the alternate would be to define an inverse relationship on the related entity back to the root which might be useful, but may result in extra response payload.

Core audio tips

Incase it ever gets removed, some Core audio gold from

  • The Audio Unit Programming Guide is required reading for using Audio Units, though you have to filter out the stuff related to writing your own AUs with the C++ API and testing their Mac GUIs.
  • Get comfortable with pointers, the address-of operator (&), and maybe even malloc.
  • You are going to fill out a lot of AudioStreamBasicDescription structures. It drives some people a little batty.
  • Always clear out your ASBDs, like this:

    This zeros out any fields that you haven’t set, which is important if you send an incomplete ASBD to a queue, audio file, or other object to have it filled in.

  • Use the “canonical” format — 16-bit integer PCM — between your audio units. It works, and is far easier than trying to dick around bit-shifting 8.24 fixed point (the other canonical format).
  • Audio Units achieve most of their functionality through setting properties. To set up a software renderer to provide a unit with samples, you don’t call some sort of a setRenderer() method, you set the kAudioUnitProperty_SetRenderCallback property on the unit, providing a AURenderCallbackStruct struct as the property value.
  • Setting a property on an audio unit requires declaring the “scope” that the property applies to. Input scope is audio coming into the AU, output is going out of the unit, and global is for properties that affect the whole unit. So, if you set the stream format property on an AU’s input scope, you’re describing what you will supply to the AU.
  • Audio Units also have “elements”, which may be more usefully thought of as “buses” (at least if you’ve ever used pro audio equipment, or mixing software that borrows its terminology). Think of a mixer unit: it has multiple (perhaps infinitely many) input buses, and one output bus. A splitter unit does the opposite: it takes one input bus and splits it into multiple output buses.
  • Don’t confuse buses with channels (ie, mono, stereo, etc.). Your ASBD describes how many channels you’re working with, and you set the input or output ASBD for a given scope-and-bus pair with the stream description property.
  • Make the RemoteIO unit your friend. This is the AU that talks to both input and output hardware. Its use of buses is atypical and potentially confusing. Enjoy the ASCII art:

    Ergo, the stream properties for this unit are

    Bus 0 Bus 1
    Input Scope: Set ASBD to indicate what you’re providing for play-out Get ASBD to inspect audio format being received from H/W
    Output Scope: Get ASBD to inspect audio format being sent to H/W Set ASBD to indicate what format you want your units to receive
  • That said, setting up the callbacks for providing samples to or getting them from a unit take global scope, as their purpose is implicit from the property names: kAudioOutputUnitProperty_SetInputCallback and kAudioUnitProperty_SetRenderCallback.
  • Michael Tyson wrote a vital blog on recording with RemoteIO that is required reading if you want to set callbacks directly on RemoteIO.
  • Apple’s aurioTouch example also shows off audio input, but is much harder to read because of its ambition (it shows an oscilliscope-type view of the sampled audio, and optionally performs FFT to find common frequencies), and because it is written with Objective-C++, mixing C, C++, and Objective-C idioms.
  • Don’t screw around in a render callback. I had correct code that didn’t work because it also had NSLogs, which were sufficiently expensive that I missed the real-time thread’s deadlines. When I commented out the NSLog, the audio started playing. If you don’t know what’s going on, set a breakpoint and use the debugger.
  • Apple has a convention of providing a “user data” or “client” object to callbacks. You set this object when you setup the callback, and its parameter type for the callback function is void*, which you’ll have to cast back to whatever type your user data object is. If you’re using Cocoa, you can just use a Cocoa object: in simple code, I’ll have a view controller set the user data object as self, then cast back to MyViewController* on the first line of the callback. That’s OK for audio queues, but the overhead of Obj-C message dispatch is fairly high, so with Audio Units, I’ve started using plain C structs.
  • Always set up your audio session stuff. For recording, you must use kAudioSessionCategory_PlayAndRecord and call AudioSessionSetActive(true) to get the mic turned on for you. You should probably also look at the properties to see if audio input is even available: it’s always available on the iPhone, never on the first-gen touch, and may or may not be on the second-gen touch.
  • If you are doing anything more sophisticated than connecting a single callback to RemoteIO, you may want to use an AUGraph to manage your unit connections, rather than setting up everything with properties.
  • When creating AUs directly, you set up a AudioComponentDescription and use the audio component manager to get the AUs. With an AUGraph, you hand the description to AUGraphAddNode to get back the pointer to an AUNode. You can get the Audio Unit wrapped by this node with AUGraphNodeInfo if you need to set some properties on it.
  • Get used to providing pointers as parameters and having them filled in by function calls:

    Notice how the return value is an error code, not the unit you’re looking for, which instead comes back in the fourth parameter. We send the address of the remoteIOUnit local variable, and the function populates it.

  • Also notice the convention for parameter names in Apple’s functions. inSomething is input to the function, outSomething is output, and ioSomething does both. The latter two take pointers, naturally.
  • In an AUGraph, you connect nodes with a simple one-line call:

    This connects the output of the mixer node’s only bus (0) to the input of RemoteIO’s bus 0, which goes through RemoteIO and out to hardware.

  • AUGraphs make it really easy to work with the mic input: create a RemoteIO node and connect its bus 1 to some other node.
  • RemoteIO does not have a gain or volume property. The mixer unit has volume properties on all input buses and its output bus (0). Therefore, setting the mixer’s output volume property could be a de facto volume control, if it’s the last thing before RemoteIO. And it’s somewhat more appealing than manually multiplying all your samples by a volume factor.
  • The mixer unit adds amplitudes. So if you have two sources that can hit maximum amplitude, and you mix them, you’re definitely going to clip.
  • If you want to do both input and output, note that you can’t have two RemoteIO nodes in a graph. Once you’ve created one, just make multiple connections with it. The same node will be at the front and end of the graph in your mental model or on your diagram, but it’s OK, because the captured audio comes in on bus 1, and some point, you’ll connect that to a different bus (maybe as you pass through a mixer unit), eventually getting the audio to RemoteIO’s bus 0 input, which will go out to headphones or speakers on bus 0.

Improving playback timing with Core MIDI Network MIDI

The network MIDI feature for OS X and iOS is fantastic – a great way to connect your mobile and desktop apps together. However mileage can vary and timing can often become unacceptable depending on the mood of your local network.

This makes sense – Wifi is not designed for sending infrequent low latency small packets where each packet must be sent instantly.

Things can be massively improved by creating a private local network to connect your iOS device to your mac directly. And in a typically Apple way it’s ridiculously easy to do!

On your mac in the wifi menu select ‘Create Network…’ and follow the instructions. Then on your mobile device switch to the new network.It may complain that there is no internet connection but that’s ok – you don’t want interruptions from email/twitter/etc during your MIDI session anyway 🙂

Reducing disk bloat on the command line

Some handy nix snippets to find which directories are eating into your disk space

Show me it all

First the daddy. Show disk free (df) on all mounts (-h is for (h)uman readable disk sizes)

Show a breakdown

Show disk usage (du) for a file/directory or set

-h Again human readable

-s Value for each file/directory

-c Show a grant total for all

Rather than specify a file or path I usually cd into the directory and use a wildcard:

To sort by size pipe to, err, sort. With the -h option it groups k, m, g together so not actually in true size order but it’s easy enough to see what’s what. Alternatively remove the -h and look at big numbers!

These commands will miss dot files by default. If you need to see them use

Show recent file changes

Want to see the preference files updated when you launched an app? Or can’t remember what you’ve been doing recently? Show files changed in the last 10 minutes.

To widen the net to longer durations, say files updated in the last 10 days use ctime

Time defaults to days but alternative units can be specified:

s: second

m: minute

h: hour

d: day

w: week

Show recent file access

For file access use either amin or atime with similar pattern.

Do  dangerous stuff

Please use without effect of medication. Now you’ve found files that are possibly clogging up space (how about old backup/cache/log files?) with care you can delete files. Warning it’s recursive. You can/should run the command without the -delete to see what’s going to happen first.

SVN Workout 1 – Branch and merge practise

RECOMMENDED TO USE LATEST SVN CLIENT, 1.8.4 (at time of writing)

Version control is only as good as how it’s used. And it can be used really badly. All the facilities are there to prevent trouble from occurring, but only if you use them.

A basic reliable version control workflow assumes that the trunk copy contains only stable production ready code.

To ensure only stable production ready code goes into the trunk, branches must be created to contain the work for each feature for a future release. Let’s say that again:

To ensure only stable production ready code goes into the trunk, branches must be created to contain the work for each feature for a future release.

It’s often tempting to just use the trunk for everything, avoid branching and merging to save time and hassle but that leads to bigger problems down the line and costs MORE time to later fix.

So to dispel the rumour that branching and merging is difficult, let’s do a little svn workout to show how easy it can be, and how much hassle it can save when real world problems occur.

Setting XDebug breakpoints for remote CLI scripts with PHPStorm

PHPStorm makes setting and using XDebug breakpoints easy. Just define the server configurations and click the phone icon to listen mode and load your site in a browser. With the correct xdebug.ini parameters it works if your application is either running on the same machine as PHPStorm or on a remote server.

Running CLI scripts on a different server to the machine PHPStorm is running on require a bit more effort though.

First ensure the phone icon is active so PHP Storm is listening for connections.

Again ensure server configurations are setup for your project. There will be a server name and configuration name.

On the remote server define the server name that PHPStorm should listen for:

export PHP_IDE_CONFIG=””

Now execute the CLI script on the remote machine.

If the server name you specify matches the server name you have defined in your project execution will halt on your breakpoints.

Or will it?

For me this works fine for some environments. When this is not enough export some more:


to get it going and


to stop it. Nice!

How to rename a mongodb collection

On installing a sample node.js application the associated mongo files ended up with long collection names including ‘.’ and digit blocks.

To rename you cannot access the collection directly, as the dot-digit combination is illegal. Instead use

to solve the problem.

rsync. The best way to copy files between servers.

rsync to copy multiple files via ssh to a remote machine. Same syntax as scp:

-a Archive mode (ensures that symbolic links, devices, attributes, permissions, ownerships, etc. are preserved in the transfer)

-v Increase verbosity

-z Use compression

–progress, to show progress

In this example all files in /root/temp on the host machine are copied to the remote machine.

Setting up an NFS share between a virtual Parallels Ubuntu server and OS X Mountain Lion

Parallels upto now has made my life very easy for virtualisation. Instances run fast, don’t slow down the mac and setting up shared directories so remote server and host can see the same application files works flawlessly. Until today that is…

Today I need to setup Symfony2. During it’s setup, config and running it uses a combination of cli and web tools which did not get on with my usual shared directory setup. I could either get it running on the mac without the ubuntu server able to see it or working on ubuntu without the IDE on the mac able to see it.

The solution turned out to be simple – NFS to the rescue!

On the mac to setup the NFS server edit the /etc/exports file to specify the directory to share and the network range that can see it. Most importantly is  -mapall=root:wheel which ensures that all client writes appear as root, so there are no permission hassles on the client end.

UPDATE: on using this method to share a system directory on another project the share failed, so it may not be possible to use this option in all cases.


At the client end create a mount point and mount:

And for good measure add into /etc/fstab

Export a mongo collection as csv

Started to create a script to do this, then realised mongoexport can do this out of the box!