I build products, teams, and companies.

Verily Life Sciences (2019-2022)

Verily is working to bring the promise of precision health to everyone, every day. Precision health leverages data from a wide variety of clinical and non-clinical sources to arrive at the best intervention for a person or community.

As head of Verily’s Medical Devices Software organization, I directed five teams of embedded, mobile, and cloud developers. These teams worked on five separate programs, each having significant scope and cross-functional collaboration with other Verily teams and external partners. We created the software that runs Verily’s highly-integrated, medical devices and connects them to the cloud. Our devices interface directly with humans to generate the data required for precision health solutions.

During my time at Verily, my teams brought three significant cloud-connected hardware and software products from early development to shipping, while also supporting a number of other research and development programs.

Verily Retinal Camera (2022)

   

The Verily Retinal Service combines an advanced camera with workflow software, making it easier for clinicians to conduct quality diabetic retinopathy screens. The lightweight and portable camera includes advanced lighting, stabilization, and Machine Learning technology that requires no dilation of the eye in most cases. These features allow a minimally-trained operator to take high-quality photos of a patient’s retina, which will make this type of diagnostic imaging more accessible.

My teams developed the camera software (C++/Python), the camera operator UI (React/JS), and the cloud back-end (Go) that managed the fleet, ingested image data and metrics, stored the data, sent it to the diagnostic services, and interacted with the workflow software built by another team at Verily.

After a reorg, I inherited this program with a small and shrinking team, an unstable proof-of-concept hardware/software platform, no back-end, and a mandate to launch the product asap. With this ignoble start:

• I rebuilt the team, starting by hiring a tech lead and an engineering manager. The tech lead evaluated the prototype Camera platform and proposed a path forward that reused as much as possible from the original design, while improving stability and readying it for production. The engineering manager focused on hiring the rest of the team and onboarding them to the program.
• I sketched a cloud architecture and socialized it with the team that was working on the workflow software so that we could begin making decisions and agree on division of responsibility. My team didn’t have a cloud tech lead for the program yet, so I stepped in to play this role to keep the program moving forward until we could land a hire.
• I fought for a sane schedule, helping leadership understand the prototype nature of the existing system and the potential consequences of rushing it to market. In the end, both leadership and engineering were unhappy, which is the hallmark of a good compromise. Joking aside, we did deliver under extreme schedule pressure, but defended space for responsible choices and good engineering.
• I directed the team through numerous decisions and tradeoffs as the program proceeded.

Despite the many challenges, we successfully launched the product with our first partner and the program continues to grow. I look forward to increased deployments of the system and a dramatic decline in preventable blindness from diabetic retinopathy.

Verily Study Watch (2022)

     

Verily Study Watch is a sensor-packed wearable device for non-invasive, continuous monitoring. The Investigational Study Watch is used in research and clinical studies, and the FDA-cleared Study Watch for ECG and Irregular Pulse Monitor is used in care collaborations with our partners.

My teams developed the watch firmware and algorithms (C++) as well as the companion mobile apps (iOS/Android). They also designed the security scheme and communication protocols between the watch and our cloud services.

Verily Study Watch was launched several years before I joined Verily, so I inherited a team largely working on maintenance. After I joined, we partnered with iRhythm Technologies to add atrial fibrillation (AFib) detection to an updated watch design, which will expand iRhythm’s offerings with a wrist-worn monitoring solution. This will be launched as the Zio® Watch. On July 22, 2022, we received FDA 510(k) clearance for the end-to-end cardiac arrhythmia solution. To achieve this outcome:

• I represented Verily engineering in initial meetings with iRhythm as we formed the partnership. As the program proceeded, I handed off the engineering relationship management to my Study Watch engineering manager, giving him a new leadership opportunity at which he excelled.
• I directed my team through new hardware and software investments in our watch platform. I contributed specific domain knowledge to solve an architectural challenge involving program space and cost tradeoffs.
• I justified additional headcount to increase our velocity to meet deadlines. We hired, onboarded, and increased velocity as promised.
• As he proved himself, I made space for my Study Watch engineering manager to excel. He did an amazing job. We hit our goals and the new development and new challenges slowed. I could see that his rapid growth was limited by the phase of the program, so I found a new opportunity elsewhere in the company and transferred him to another team for an exciting new challenge. This allowed me to expand the scope of another manager on my team, who fluidly transitioned into the new growth role.

Verily Patch (2021)

     

Verily Patch is an adhesive sensor patch that regularly measures an individual’s body temperature. The device uses Bluetooth Low Energy (BLE) to transmit data through the corresponding mobile app and to Verily’s cloud back-end for easy tracking of a user’s latest temperature reading. The mobile app can alert the wearer if their temperature rises above a threshold. Temperature is one of the most common signals of changes in health—by collecting temperature data frequently and accurately, Verily Patch is designed to support care broadly. The current model can be used for up to three months, the average length of the peak flu season.

My teams developed the Patch firmware, the iOS/Android mobile apps, and the cloud gateway that managed device security and ingested data. Verily’s leadership prioritized this as a high-priority program in response to the COVID-19 pandemic in 2020. My team was busy with other programs at the time. To deliver on our business objectives:

• I began the proof-of-concept firmware development myself, which let us evaluate feasibility and generate partner interest. I wouldn’t typically recommend that a senior engineering manager gap-fill with IC work; however, I knew that I had the bandwidth to both code and build the team and that, when it came to it, I prioritized building the team.
• I developed and executed a plan to roll a manager and developers off of lower-priority programs and have them kick off the production Patch software effort.
• I recruited additional contributors from across the company to complete the team.
• I matched individuals with growth opportunities, which helped with recruiting, morale, and motivation on this intense program.

Our rapid action took this program from concept to production in less than a year. While the program was ultimately shelved as the pandemic waned and Verily focused its efforts on other businesses, results from our initial study were promising.

The Patch photos above are courtesy of Jon Grossman, who led the Patch design.

Fresh Health (2017-2018)

In 2017, I joined Mike Hanuschik in creating the consumer healthcare startup, Alpine Oral Tech, Inc., which later became Fresh Health. Fresh Health is developing the Proclaim Custom-Jet Oral Health System that has the potential to help billions live healthier, longer lives. I helped Mike bootstrap the company, hire our first engineers, and develop patents and prototypes to secure additional funding. While I was largely working in a leadership/management capacity, I got hands-on where needed. Specifically, I enjoyed the following eclectic projects:

• I helped set up our Google Cloud project where we were running computational fluid dynamics simulations for jet development.
• I designed and built a hardware/software experiment platform capable of controlling high pressure fluids while acquiring data about pressure and flow. For this, I designed and manufactured a custom PCBA and developed experimenter-friendly software in Python.
• I invented a shear force surrogate material that we could reliably and repeatedly apply to teeth models to determine whether our jet designs were improving. The material allowed for consistent application and photographed well so that we could easily measure and quantify our results.

Watch for developments at https://freshhealth.com

Mindtribe (2001-2017)

     

2005

2007

2015

I joined Mindtribe as the sixth employee in January of 2001. Over the subsequent 16 years, I helped build Mindtribe into a 40 person, world-class engineering firm. As a partner and vice president, I was responsible for business development, sales, engineering management, program management, process development, strategy, hiring, and mentorship.

During my time at Mindtribe, I figuratively wore a number of hats (few literally as I’m not a hat person). Being a small team, we each had to sport a “get things done” attitude. In addition to engineering, writing proposals, and fielding sales leads, my early role at Mindtribe included managing our IT, building our websites, creating documentation templates and process tools, acting as project librarian, cleaning the office, making coffee, and ensuring that everyone got a cake on their birthday.

As Mindtribe grew, I grew along with it. I became a more effective leader. I learned how to work better with others and amplify their abilities. I became more confident and less afraid to give or receive feedback. I learned that I didn’t need to do everything myself and handed off responsibilities for which others were more effective. I developed deep engineering skills and insights spanning numerous technologies and industries.

I now enjoy applying what I learned at Mindtribe to grow new companies.

Human Interface Device for Surgical Robot (2016)

A medical device startup is creating an innovative surgical robotics system. They engaged Mindtribe to create a medical-grade input device that surgeons will use to control the robots.

I led the first half of this ongoing program, overseeing a team of mechanical, electrical, and firmware engineers working towards a production-intent prototype. We collaborated with the client and an industrial design firm to create experiential prototypes to narrow down the form factor and ergonomic concerns. In parallel, we gathered system requirements, performed an FMEA to understand how we were going to address safety issues, researched sterilization solutions, and kicked off the system architecture.

Interesting technologies we researched during this program included industrial communication protocols such as EtherCAT, sterilization methods such as ethylene oxide and STERRAD®, and electrical and firmware-level error mitigation/redundancy. The system was built with an NXP Kinetis K65 ARM Cortex-M4F microcontroller.

I left Mindtribe during this program, so I transitioned it to another Mindtribe VP. Our client had originally planned to take the design into production themselves; however, our work impressed them and, after I left, they extended Mindtribe’s involvement to help them through production.

Wine Preservation Device (2016)

Frustrated with the expense and ineffectiveness of existing consumer-oriented wine preservation solutions, a startup took an innovative approach to the problem. Mindtribe to helped them refine their existing prototype and prepare it for production.

I led a team of electrical and firmware engineers to evaluate and redesign their existing prototype. The product creatively uses a minimal set of sensors to observe the ambient conditions and chemical reactions that impact wine storage. We needed to unravel a complicated set of possible conditions and events to create robust, production-intent firmware. The product was powered by a single coin cell, so we had to ensure that both the electrical and firmware systems were designed for ultra low power. This system was built around an NXP Kinetis KL03 ARM Cortex-M0+ microcontroller.

Mobile EKG (2016)

A medical device startup asked Mindtribe to help them develop a new version of their existing mobile EKG product.

I led a team of mechanical and electrical engineers to take the new design to production. We identified and solved several signal integrity and power management issues, which ultimately led to the product’s success. We developed a factory test procedure and traveled to Shenzhen, China where we worked with an injection molder and contract manufacturer to take the design through a successful production build.

Encrypted Communications Device (2015-2016)

A startup with deep security expertise was building an iPhone accessory that secured voice calls with strong encryption. Because of our extensive experience with iPhone accessory development, they reached out to Mindtribe to help them debug a problem with their communications stack.

I began this program by leading a team of firmware engineers to diagnose and solve our client’s firmware issues. We quickly squashed a number of bugs and demonstrated an end-to-end encrypted call using their hardware prototypes. The client then expanded our scope to help them finish the product development and take it through production. I became their acting VP of Engineering and managed a combination of our team, their team, an electrical consultant, an audio consultant, an app development firm, a security auditor, and a contract manufacturer in the Philippines. Together, we refined the electrical, mechanical, and firmware designs through a series of prototypes. Due to the product’s mechanical complexity, security requirements, and the required level of polish, we needed to closely support the contract manufacturer. I repeatedly traveled to Manila with my team to help the manufacturer through a series of builds over six months.

I worked with the startup’s CEO and sales team to plan the program around their launch and fundraising needs. Because they lacked in-house technical critical mass, I also helped them with their hiring efforts by screening and interviewing for them. When Mindtribe disengaged, I left them with an engineering team capable of sustaining and evolving the product line.

During this program, I had the opportunity to dive deep into some interesting technologies such as encryption (AES, elliptic curve, and Diffie-Hellman), random number generation, realtime audio CODECs and DSP, hardware security, injection molding, and ultrasonic welding. The system was built with an NXP Kinetis K61 ARM Cortex-M4F microcontroller and a Cirrus Logic DSP.

The Factory Provisioning Station for Encrypted Communications Device project below details more of our factory work for this program.

Factory Provisioning Station for Encrypted Communications Device (2016)

The Encrypted Communications Device project included a complicated cryptosystem that required the generation of a unique identifier and cryptographic keys for each device’s printed circuit board assembly (PCBA). This provisioning step needed to be performed securely on the manufacturing line in Manila. For early engineering test builds at the contract manufacturer, my engineering team was able to manually perform the provisioning step and load the device firmware; however, we needed to build a station that minimally-trained operators could use during production. Through the contract manufacturer, I commissioned an electromechanical fixture to hold the PCBAs and make the appropriate connections to the product’s microcontroller. It was up to my team to write the provisioning software that would use this fixture to allow operators to program PCBAs.

As our manufacturing build date drew near, it became clear that my team was understaffed and wasn’t going to be able to complete the provisioning station software in time for our first production test build. Manually provisioning units would have been incredibly painful for my team and would have prevented us from achieving our goal of exercising the production-intent line. Because of this, and because I had relevant expertise, I jumped in and wrote the required software—largely during flights to/from Manila for other builds. The software:

• Provided an operator-friendly user interface
• Was capable of provisioning up to 4 PCBAs simultaneously
• Was secure—as defined by our client’s security officer
• Could send encrypted provisioning results and statistics to our client

I wrote the software in Python and used Kivy for the user interfaces. Data was stored in SQLite databases and encrypted for transport between the factory and our client using gpg2

Ambient Mobile Charging (2015)

A startup offering mobile charging as a service was having quality problems with their devices. The service operates via a deployment of smart batteries throughout urban areas. When users encounter these smart batteries and connect their phones, they’re prompted to sign into the startup’s service to pay for the battery usage. Unfortunately, the batteries would not reliably connect to various Android phones. The issues seemed to be a combination of firmware and app-level bugs. Because of our prior Android accessory and app experience, Mindtribe was asked to help debug the issues.

I managed this program and planned/oversaw a short sprint during which my team debugged the issues while I lent USB expertise to the program.

Medical Device BLE Bootloader (2015)

Mindtribe helped a medical device startup create an over-the-air (OTA) firmware update mechanism for their Bluetooth LE (BLE) device.

I managed this program and planned/oversaw a development effort to create a bootloader, BLE OTA update protocol, and a macOS test application. While my team performed the entirety of the implementation, I assisted in the architecture and quality assurance of our deliverables.

Illuminated Dashboard Moustache (2014)

     

Lyft, the popular ride sharing service that is headquartered in SF, engaged Mindtribe to work with Ammunition to overhaul Lyft’s iconic, fuzzy “carstache”. Our mission was to create a smaller, sleeker, more refined emblem that maintained the superb brand recognition Lyft had earned with their standout ‘stache.

We began by supporting Ammunition’s concept creation/refinement phase through a series of studies and prototypes to understand safety concerns, costs, and feasibility.

Once Lyft and Ammunition selected the final “glowstache” concept, Mindtribe created the detailed mechanical, electrical, and firmware design. The glowstache was effectively a very fancy flashlight. It had an internal battery that recharged via a USB port, fading and blinking behaviors to indicate charging status and to allow the driver to signal a fare, a magnetic dash mount, and a diffuse glow in Lyft’s signature, hot pink color. Though this was a relatively simple product, achieving a very exacting and refined industrial design is a time consuming process—and Lyft’s timeline was aggressive. To further complicate matters, the color pink comprises red and blue. By law, cars may not have forward-facing red lights and blue is reserved for emergency vehicles. So, for both branding and legal reasons, we needed to ensure a very precise color match that didn’t trend either too red or too blue under both transmitted and reflected lighting.

Mindtribe located a combination of overseas injection molders and domestic contract manufacturers. We traveled to China to meet with the injection molders to reduce iteration times while ensuring quality. We located a color specialist in Los Angeles and flew down to sit and test colors with them. Through a combination of creativity, skill, and brute force, we went from concept to production in just four months.

I planned and led the manufacturing phase of this program. My mechanical expertise is weak compared to my software and electrical expertise, so I found this mechanically intense program to be a skill-building and confidence-inspiring experience. During the course of the program, I noticed that my team was struggling with the problem of even illumination within the small volume of the glowstache. I jumped in and helped out with a deep dive into light spreading technology and discovered the solution that we used in production.

Microcorruption Capture The Flag (2014)

Matasano Security (now part of NCC group) and Square teamed up to create Microcorruption: an awesome Capture the Flag game that involves reverse engineering MSP430 assembly to unlock a virtual lock. Each level gets progressively harder as the fictitious lock vendor patches security holes and introduces new ones.

While this was more fun than professional, I used my reverse engineering skills, microcontroller knowledge, and understanding of security exploits to complete the game and made it into the top 100 list in the Microcorruption hall of fame.

Microcontroller Platforms (2013-2017)

Mindtribe is always looking for ways to build products faster. One such method is to invest in intellectual property development that’s ready for reuse on projects. To this end, we created several microcontroller platforms with electrical reference designs and production-oriented firmware codebases.

I led programs to create the following platforms:

• Corgi - A general-purpose microcontroller platform based upon the TI TM4C12x (formerly Stellaris) Cortex-M4F processor line. We created a microcontroller motherboard that hosts daughterboards called “pups”. Inspired by Arduino shields, the pups extend the platform’s functionality with sensors, actuators, battery chargers, etc. The schematics/layouts are created in Altium using our production workflow, so reusing the work is trivial. I personally created a firmware codebase that demonstrates a driver model with example programs and supports a production build system with unit tests. This platform became an important part of Mindtribe’s hiring and onboarding efforts. We use the system in interviews and new hires extend it to learn our development and production workflows.
• Flea - A Bluetooth LE microcontroller platform based upon the Nordic Semiconductor nrf51822 BLE microcontroller. Schematics and code were built according to our production workflow. This was a highly leveraged platform that kickstarted multiple client programs.
• CARL - A high-end microcontroller platform based upon the NXP i.MX6 ARM Cortex-A microcontroller. This platform runs Linux and is capable of video processing and streaming applications.

Automotive Gestural Input Device (2013)

A large automotive manufacturer wanted to explore gesture-based input in an automotive application.

I structured and ran a development effort to prototype an input device that uses a unique touchpad to convert gestures into alphanumeric input. The prototype communicated with the touchpad via USB and performed gestural processing in Python, using Kivy to provide a GUI that visualized the results.

Human Interface Device for Next-Gen TV Concept (2013)

A large consumer electronics manufacturer wanted to explore improvements to TV remote controls and on-screen user interfaces.

I structured and ran a development effort to prototype a new remote control that incorporates capacitive sensing elements. We partnered with local user experience design and development firm, Stimulant, who developed the on-screen TV experience that our hardware controls. To accelerate Stimulant’s development and integration of our work, we built an Android-based remote control simulator that they used while our hardware was under development. Once our hardware was ready, it took just a few hours to replace the simulator with the actual hardware and re-tune the system.

Smart Building Management System (2013)

   

Verdigris partnered with Mindtribe to develop a system that uses artificial intelligence to improve commercial facilities management. Mindtribe built Verdigris’ hardware platform while they focused on the backend and application.

When the teams began to encounter quality issues, the program manager brought me in to help diagnose and fix the problem. I discovered that the complexity of the system did not lend itself well to structured, end-to-end testing, so the teams were taking longer than usual to pinpoint and fix bugs. I recommended and helped architect a system that allowed for better testing, which improved the development workflow.

As Verdigris began their hiring efforts, I provided input to help with their job postings and their interview process.

Automotive Apps (2012)

Ferrari, faced with increasingly-demanding expectations around infotainment functionality and user interface, wanted to explore how to modernize their infotainment experience by focusing on better integration with Apple products. Apple put Ferrari in touch with Mindtribe because of our reputation for having helped several major automotive manufacturers more closely integrate with iOS devices. Mindtribe, along with our design partner, Ammunition, helped Ferrari define a new system architecture and a flagship iOS app around which they could showcase their new technology.

Having deep experience with iOS accessory integration, I led the technical portions of our workshops with Ferrari to help their management and technical teams understand and overcome the challenges associated with this technology.

Mobile Drawing Application and Hardware Tools (2012)

     

Adobe engaged Mindtribe to help them explore the use of hardware to enhance the mobile drawing experience and bring back the familiar experience of drawing with pen and paper. We built a connected stylus capable of producing beautiful strokes on an iPad. To support this effort, Mindtribe wrote an iOS app to test and demonstrate the drawing capability of the stylus. During this program, Adobe asked us to also explore the concept of a hardware ruler for technical drawing. We prototyped this ruler and extended our test app with new functionality to support it. Around this time, we were collectively coming to the conclusion that these prototypes were more than intriguing concepts—they were potentially-shippable products.

Adobe decided to proceed with the productization of the pen and ruler, to be sold as Adobe Ink and Slide. They felt that evolving our test app into a polished drawing app as a skunkworks project would be a faster path to a showcase app for the new drawing accessories than trying to get the hardware support integrated into Adobe’s existing apps.

We dedicated Mindtribe developers to the application and I stepped in to manage it as its own program—codenamed Project Parallel. I worked with Adobe’s project manager to put together a production schedule that aligned with the hardware release. Together, we ran the program using an agile methodology. The team met weekly to plan each sprint and delivered features by demonstrating them at the subsequent planning meeting. The iOS developers worked closely with Adobe’s designers and Mindtribe’s hardware team to reduce integration cycles to hours. We moved blazingly fast.

As the program progressed, we added iOS developers from Adobe to increase our velocity and incorporate Adobe technologies such as Creative Cloud. I engaged Adobe’s internationalization team to help us plan early for internationalization and to perform the actual localization for the countries in which the product would launch. I later engaged Adobe’s quality assurance resources to help us set up automated UI testing and to perform manual application testing with the hardware.

Early in the program, I contributed to the iOS development itself; however, I was lucky enough to have an exceptionally capable team and many Adobe resources, so I was able to step back and focus on removing all program-level obstacles from their path.

Together, Adobe and Mindtribe shipped a beautiful and capable product called Adobe Line that became a model for app development within Adobe. We met with other iOS teams within Adobe and shared our experience and methodology.

Secure, Mobile Point of Sale System (2012)

Square kicked Mindtribe off on their Point of Sale system design as we were wrapping up their Secure Mobile Card Reader. The system, known as the Square Stand, is intended to appeal to brick-and-mortar stores that need a more professional solution than the Mobile Card Reader offered. The Stand leverages an iPad for its display and connectivity and allows users to connect barcode scanners, cash drawers, and receipt printers through its USB hub.

I structured and ran this program from concept through manufacturing. By this time, Square had a sizable mechanical engineering team and a growing electrical team. Mindtribe led the firmware development, contributed to the electrical design, and supported the mechanical design. I provided cross-functional engineering management to keep the efforts in sync—often collocating at Square’s office and working with Square’s new VP of hardware to understand and manage the requirements and development effort.

I contributed significantly to the firmware itself, kicking off the production codebase that ran on an ST STM32F207VE ARM Cortex-M3 microcontroller. The Apple iPad does not directly support accessories such as receipt printers, so our firmware’s primary job was to act as a USB host for the peripherals and shuttle data between them and the iPad using Apple’s proprietary accessory protocol. We also advanced our work from the Mobile Card Reader project and built a new, secure, dual-sided magnetic stripe reader for the Stand.

I oversaw the work performed by Mindtribe and Square, managed an iOS consultant who helped us develop a stand SDK and test app, managed multiple hardware/software vendor relationships, and managed the Stand’s security auditing. As we transitioned the Stand to production, I helped Square set up the manufacturing line at a contract manufacturer in Mexico, flying down repeatedly to ensure smooth builds to keep us on schedule.

During the Stand development, I helped Square interview, hire, and train their first firmware developers and left Square with a team that was capable of sustaining and evolving the product line.

Secure, Mobile Credit Card Reader (2011)

   

Square engaged Mindtribe to help them create a secure and more reliable version of their mobile credit card reader. We helped them take this from concept through production. The design needed to be secure, but very low cost and very low power.

I structured and ran the firmware development effort for this reader. I worked with Square to understand the security requirements and kicked off the production codebase. I then handed it off to my team and managed the remainder of the development, overseeing our work and providing technical expertise.

We managed to pack magnetic stripe signal capture, signal decoding, AES encryption (with chaining, message authentication, and a nonce), tamper resistance, and data transmission into a tiny TI MSP430 with 8KB of flash and 256 bytes of RAM. We developed the encryption code with outside security experts and had them audit our work for security purposes.

We worked with Square’s security team to set up a factory programming workflow in China, including secure key injection using a hardware security module.

Desk Scheduling Device (2011)

A design company engaged Mindtribe to help them develop a prototype for a desk scheduling device for office environments with shared desks. This was to be a small, low power, cost-effective, networked device that could be deployed to hundreds of desks in large enterprise environments.

I structured and managed this program. I worked closely with the client to understand and capture the requirements. Working with a project manager, I put together schedules and budgets for the program and worked with the team to evaluate unit costs as we underwent component selection.

We built a functional prototype that drove a display and communicated via Zigbee. We used a Renesas RX62N microcontroller, which offered a means of driving a TFT display without the expense of a full TFT interface.

Automotive Controls for iOS (2011)

Having successfully demonstrated iPod Out, the Toyota InfoTechnology Center USA asked Mindtribe to help them add new input devices to their system that would allow drivers more control over the iPhone.

I structured and led the effort to write firmware for a Cypress CY8C24894 PSoC that connected to Toyota’s head unit and sent button information via USB.

Mobile Video Communications (2010)

A large electronics firm engaged Mindtribe to help them create a prototype of a mobile video communications device.

I helped manage the program, specify the system architecture, manage an application development consultant, and write video streaming firmware. We used a TI TMS320DM368 DaVinci digital media processor running Linux. We built drivers for the camera, LCD, and peripherals. We created a Linux BSP and SDK that included QT for the application developers and we integrated the newly-released SkypeKit SDK to support Skype video calling over WiFi.

Solar Battery Charger (2010)

A startup engaged Mindtribe to help them build a portable battery system for developing countries. The system uses solar power to charge a battery and allows users to make small mobile payments to charge their devices from the battery.

I provided firmware oversight and mentoring for the client’s firmware development efforts.

iPod Out Automotive Integration (2010)

Upon learning that we helped develop the technology, Toyota InfoTechnology Center USA asked us to help them integrate iPod Out into a prototype vehicle so that they could understand and experiment with it.

I structured and led the effort to help Toyota integrate Mindtribe’s MFi Library into a vehicle to demonstrate iPod Out functionality. This involved helping them port the library to their head unit and working with them to bring up the system.

iPod/iPhone Communication Library (2010)

As I saw increasing demand for Mindtribe to develop smart iPhone accessories—especially in the automotive space—I decided that we should invest in wrapping up our work to date into a licensable library so that we could accelerate accessory development programs.

I built a highly portable C library and C++ reference application that implemented the entirety of Apple’s massive Made for iPhone (MFi) specification.

Gestural Input Device Concept (2010)

An input device company asked Mindtribe to help them explore a novel way of providing low-cost multi-touch input using capacitive sensing elements and a traditional single-touch touchpad.

I structured and managed a program to develop firmware for a Cypress CY8C24894 PSoC. This firmware enumerates as a USB HID device and combines capacitive sensing elements and touchpad data to emulate a multitouch input device.

Phone-controlled Tetromino Game (2010)

When Mindtribe moved into a new office on University Avenue on Palo Alto, we found ourselves with a large, very visible window. We decided to create an interactive sign that would advertise our existence and engage passersby.

Working with a graphic designer, I built a tetromino-style game in Adobe Flash. I wrapped this Flash game with a Windows application that connected to a modem, answered calls, provided DTMF control over the game, took a photo of the player using a webcam, and posted high scores to our website.

The high scores were keyed to a player’s caller ID. I built out a web backend that allowed users to “claim” their scores by entering their phone numbers and providing their names. These went into a moderation queue before being publicly displayed on our website.

Sleep Apnea Solution (2009)

Apnicure engaged Mindtribe to develop a better treatment for sleep apnea that is quiet and allows users to breathe naturally through their nose without being restricted by a mask. Mindtribe worked with ID and mechanical engineering firm, Whipsaw, to create this product. Mindtribe developed the electrical and firmware designs and supported Whipsaw on the mechanical design.

I structured and ran the firmware and application development for this product, managing our team and an additional consultant. We built firmware that runs on an Atmel AT32UC3A1256 microcontroller, which performs the pump control, data logging, and communications. We also wrote firmware for a Cypress CY8C21334 PSoC to perform capacitive sensing for fluid level detection. We created several C# Windows applications designed to allow physicians to configure treatment parameters and retrieve usage logs.

Mobile Notification Device Concept (2009)

A large consumer electronics company was creating a smart watch that would allow users to see incoming call and text information on their wrists. They asked Mindtribe to help with specific technical investigations and proofs of concept.

I helped develop OLED display drivers and optimized routines for scrolling and navigating graphical data on an OLED from a very resource constrained microcontroller. I delivered C code that demonstrates these routines, running on TI LM3S9B96 and LM3S1811 microcontrollers.

Bluetooth Mobile Notification Device (2009)

   

Before the era of smart watches, a startup had an idea to create a mobile notification device that would allow users to see incoming call information and put calls on hold without digging their phones out of their pockets or purses. This startup worked with Mindtribe to help refine their prototype for fundraising purposes.

I managed and contributed to the development of the product’s firmware, which runs on a CSR chipset designed for Bluetooth communications.

Architectural Workshop for Smart Toy (2009)

When Sifteo began building their interactive gaming platform, they asked Mindtribe for product development input. We put together a workshop to help them detail out the product features, understand requirements, and begin to flesh out a system architecture.

I led the firmware portion of this workshop and followed up with specific architectural help as questions arose.

Medical Temperature Monitor (2009)

Medical device startup, Arizant (now part of 3M), had developed an innovative, non-invasive way of measuring core body temperature. They engaged Mindtribe to productize their technology in the form of a medical temperature monitor.

I structured and ran the firmware development effort for this program. I also contributed a C graphics library for drawing to the graphical display from the product’s LPC1313 Microcontroller.

Media Tablet Software Platform Research (2009)

A large telecommunications provider was interested in developing their own media tablet for content delivery (similar to a set-top box, but mobile). Since they were not familiar with hardware development, they asked Mindtribe to help them architect the system.

The client was specifically interested in what operating systems would be appropriate for such a device and how they might go about enabling 3rd party applications. I put together a body of research and a presentation to educate them about the options.

Automotive iPhone Integration (2009)

Anticipating a need for agility in automotive environments to keep up with the constant evolution of consumer smartphone features, the BMW Technology Office approached Mindtribe to help architect a new software platform that would rapidly integrate smartphone innovations into the automotive user experience. This effort involved two major components.

First, both Apple and BMW had identified that head unit navigation and control of a music on a user’s phone was a major pain point for both users and automotive manufacturers. Apple had previously exposed a massive API to allow accessories such as head units to browse a music collection on a phone, retrieve album art, control playback, etc. This API was difficult for manufacturers to consume, so the resulting experience was less than ideal and required users to learn a new, unwieldy, interface for browsing their music collection.

Apple, BMW, and Mindtribe collaborated to develop what became known as iPod Out. This specification allowed an iPhone to “push” a user interface directly to a head unit’s display over a video link. The iPod Out functionality that launched with iOS 4 gave automotive manufacturers a native Apple music interface through a much simpler API. It allowed users to control their music via the Apple user interface with which they were already familiar.

The second element that allowed for tighter phone integration in the car was a communications channel between apps running on the phone and the car’s head unit. BMW and Mindtribe worked to develop a TCP/IP-based logical tunnel over the phone’s physical USB connection to the car and a corresponding protocol specification that would allow the head unit to exchange information with iOS Apps. While our initial focus was primarily iOS, we simultaneously considered and proved feasibility on Android. This functionality opened up a new generation of applications such as live driving tutorials that can access the car’s information as well as use the phone’s sensor suite and network connectivity to tell you how you could improve your driving.

For these efforts, I created an iPhone accessory protocol stack, Linux communications simulators, and proof of concept demonstration applications. I worked with an iOS consultant to develop the tunneling technology for the app/head unit data exchange. I worked with BMW’s head unit supplier to capture requirements so that these new technologies could be put into production.

Room Scheduling System Redesign (2008)

Steelcase embarked on an effort with IDEO to redesign their popular, but aging RoomWizard room scheduling device. This networked device was built around an older x86 architecture and included a hard drive that was prone to failure over time. Steelcase asked Mindtribe to re-architect the electrical and software system.

I led the software architecture. I determined that rewriting the software was not a worthwhile investment, considering the size of the effort. I delivered a report on how to reuse as much of the existing Java application as possible with the new processor platform. I also helped investigate and architect the addition of Zigbee for control of 3rd party devices from the RoomWizard.

Industrial Positioning System for Mining Applications (2008)

A leading industrial geopositioning company engaged Mindtribe to develop a computer vision system prototype as part of an ongoing R&D effort to add advanced navigation capabilities to their existing systems. Mindtribe built a system capable of recording and storing GPS-synchronized, high-resolution video from multiple cameras.

I helped configure a rugged computer with the exceptional memory and storage bandwidth needed to capture and record video from multiple cameras simultaneously. I also wrote firmware for an NXP LPC2148 Microcontroller that manages the synchronization of the camera triggering with a GPS. The firmware transmits timestamped information via USB to the host PC that captures and records video.

CAN Stepper Motor Controller (2008)

A medical device company building protein analysis tools engaged Mindtribe to help them build a CAN-based motor controller that would be used to drive multiple motors within their products.

I structured and managed this program, which involved designing the motor control boards and writing firmware for a Microchip PIC18F2682 Microcontroller to drive stepper motors based upon commands sent over a CAN bus. Though I largely played a management role, I also lent firmware expertise and helped out with the development.

Chemical Sensing System (2008)

A chemical sensing technology startup engaged Mindtribe to help them productize their technology for detecting minute traces of harmful chemicals.

I structured and managed this program, which included an electrical and firmware effort to create a battery powered prototype capable of using our client’s technology to detect harmful chemicals and wirelessly transmit status to a base station. We built an embedded Linux platform with a DSP co-processor capable of driving the MEMS sensor and acquiring/processing data from it.

Wearable (2008)

A startup engaged Mindtribe to design a new wearable that integrated more sensor, storage, communication, and display technology than any other wearable on the market.

I managed the development of product requirements with the client and oversaw the electrical and firmware system architecture design.

Sports Watch (2008)

An electronics manufacturer was building a new version of sports watch that worked with the Apple iPod to control music and display sports activity information. The manufacturer was unhappy with the performance of their existing product’s design, so they engaged Mindtribe to help with the new version.

I wrote firmware for a Silicon Laboratories C8051F331 microcontroller that handles the sensors and watch display and transmits data wirelessly to the iPod using a Nordic nRF2402 radio. The firmware meets Apple’s strict wireless communications timing specification and achieves the very low power consumption required for the battery-powered application.

Configuration and Diagnostics Application for Implantable Medical Device (2007)

   

NovaShunt was building an implantable peritoneal pump and worked with Mindtribe on the pump design, including wireless charging and communications.

I structured and managed a software development effort to create a C# Windows application that exercises the pump’s features, performs diagnostics, and allows developers to configure pump parameters.

Point and Shoot Video Camera (2007)

     

Pure Digital engaged Mindtribe to create a new, higher end version of their point and shoot digital video camera. This product existed before mobile phones were capable of video recording and was wildly successful due to its ease of use and intuitive software for editing and sharing video. Mindtribe was responsible for taking the new hardware design from concept through manufacturing while Pure Digital focused on the application and backend software.

I structured and ran the firmware development effort, while also writing firmware myself. We were developing on a COACH 10 microprocessor from Zoran. This was the first revision of this microcontroller capable of recording video at the framerate and resolution that Pure Digital required, so we were highly incentivized to use it even though it was newly-released and unproven. We completed our electrical design based upon the new chip before the vendor had finished their own development board, so our first firmware task was to bring up a new BSP on our own hardware and debug it with the vendor—effectively proving out their silicon and software along with them. I got the video pipeline working and wrote drivers for all of the peripherals, including the imaging sensor and LCD.

We wanted to use a new, high quality LCD that wasn’t directly compatible with the TFT interface on the COACH 10, so I created a smart signal adapter using a Lattice CPLD programmed in Verilog HDL. This converts the TFT signals to drive with the LCD and injects commands into the pixel stream to control the LCD’s horizontal and vertical sync.

I also wrote the firmware for a Cypress CY8C21434 PSoC that handles both system power management and capacitive touch inputs for the user interface.

3D Human Interface Device Concept (2007)

Having achieved success in the gaming market with their 3D Mouse, Sandio began exploring new 3D input device concept that used a fusion of multiple touchpads to provide precise and intuitive 3D control of virtual objects.

I structured and managed the electrical and firmware development effort for this program. The firmware was written in C for a Cypress CY7C64215 microcontroller. It appears as a USB HID device and translates the touchpad input into 3D control.

Automotive Head Unit Proof of Concept (2007)

An innovation team at a large automotive manufacturer was exploring a new head unit concept. They asked Mindtribe to help integrate a new LCD, touch panel, and buttons onto an existing head unit computer. Mindtribe was responsible for the electrical design, mechanical integration, and firmware for a microcontroller used to adapt the new inputs for use with the head unit’s computer.

I structured and ran the firmware development effort for this program and helped with the system architecture. The firmware was written in C for a TI MSP430 microcontroller.

Robotic Hand Control (2006)

A company was working on a robotic hand prototype. While they were versed in kinematics and high-level motion control, they were unfamiliar with low-level programming and communication protocols.

I wrote a C++ application for windows that communicates with a motion controller over RS232 and uses high-level motion control scripts to drive the hand prototype.

Wireless Gestural Transfer Concept (2006)

A design firm asked Mindtribe to help them explore the idea of transferring data with a gesture. The concept was intended to simplify the act of transferring a digital business card or a file between two people in close physical proximity by using a gesture to initiate the transfer.

I wrote C firmware for a Freescale (now NXP) MC9S08GB60 microcontroller that uses an accelerometer for gesture recognition and transfers data between devices using a Freescale MC13192 radio.

Chipset and Reference Design for 3D Human Interface Device (2006)

Having achieved success in the gaming market with their 3D Mouse, Sandio wished to wrap up their IP from this product into an easily licensable design.

I worked with Sandio to select a cost-effective reference chipset based upon the Freescale (now NXP) MC9S08GT8A, the Cypress CY7C64215, and the Avago ADNS-6010 imaging sensor. I created an SDK and documentation to make it easy for 3rd parties to manufacture Sandio’s reference design or integrate the technology into their own designs.

Cashless Arcade Game Payment System (2005)

Ideal Software Systems makes cashless payment systems for the entertainment industry. They approached Mindtribe looking to overhaul their existing hardware’s industrial design, reduce system cost, and add features. The device electrically interfaces with arcade games to replace coin acceptors and ticket dispensers. There’s no standard for these and the build quality varies wildly, so our client’s device had to offer configuration options that would handle numerous voltages, poorly-grounded signals, transients, etc. The device also needed to survive attempted vandalism and having Skee-Balls chucked at it.

I helped architect the new system and led the firmware development effort. The system was built around a Rabbit 2000 microcontroller and we wrote firmware in C. One of our cost reduction measures was to replace a very expensive RS232 barcode scanner module with an inexpensive, largely mechanical module that did no onboard processing. This meant we needed to do the signal acquisition and barcode processing ourselves. Ease of barcode scanning and low cost were both key to the adoption of this system in arcades. Low cost meant that the cashless play cards were printed using extremely cheap technology, so the barcodes were of poor quality–often with gaps or blurred edges. I developed my own algorithms to maximize scan success by applying various error corrections to the scanned signal. With these, we were able to achieve scan success rates better than the original, expensive barcode module (as determined by user tests) at a fraction of the cost.

Along with barcode scanning, I wrote firmware to precisely measure and generate signals to fool the arcade games into thinking they were receiving quarters or dispensing tickets at the appropriate times and to display status on the device’s graphical LCD.

The device also participated on a LAN to exchange data with Ideal’s backend database. I wrote all of the the networking firmware for the new system and worked with Ideal to ensure the devices behave well when hundreds are deployed onto the same network.

Printer Paper Profiling Test Station (2005)

Mindtribe developed a test tool for a large printer manufacturer. This tool profiled and characterized specific aspects of a new printer paper they were developing.

I led a firmware development effort to write firmware for a Rabbit 2000 microcontroller that drives stepper motors, samples photosensors, processes and logs data, transfers data to a PC via RS232, and provides a graphical user interface via an LCD and keypad.

OLED Keyboard (2005)

   

Input device startup, United Keys, asked Mindtribe to create a prototype of a keyboard that features keys with context-aware OLED screens embedded in them. This very early precursor to the Touch Bar on Apple’s current Macbook Pro allowed the keyboard to provide application-specific functions. Mindtribe created a functional prototype for United Keys’ investor demonstrations.

I wrote C firmware for an Atmel AT91SAM764 ARM microcontroller to drive the LCDs and enumerate as a composite USB device that includes a keyboard and CDC interface (virtual serial port). The CDC interface provides the mechanism for exchanging image files to be displayed on the LCDs. I wrote a C++ Windows application to convert images to a format usable by the keyboard. I also wrote a C++ Browser Helper Object (BHO, aka Internet Explorer plugin) that our client used to demonstrate the keyboard’s functionality. The BHO changes the functionality of the context-aware keys based upon the website that the user was currently visiting.

Portable Network Analyzer (2005)

The manufacturer of a portable network analyzer worked with Mindtribe to redesign aspects of their handheld analyzer.

I structured and led an electrical effort to redesign the network analyzer to make it more suitable for production. While I largely played a management role, I helped with electrical component research and gave feedback about the electrical architecture with an eye towards easing the client’s firmware development effort on the new platform.

Warfare Training System (2005)

SRI International built a system called DFIRST™ that uses GPS satellites, wireless communications, and digital terrain map displays to create an instrumented environment for combat unit training. This system allows a unit to run through a training exercise and then analyze it later in a video-game like setting. SRI awarded Mindtribe “Subcontractor of the Year” for our work in helping them realize this system.

I led a large development effort to create firmware for multiple interface systems that gather and fuse disparate sensor data and transmit it wirelessly to a base station. The sensors include GPS, accelerometers, gyroscopes, and inclinometers. Each interface system runs on battery power and provides a local UI for configuration and diagnostics. The systems incorporate numerous processors including PIC microcontrollers and x86 single board computers.

Biometric USB Flash Drive Concept (2005)

Biogy, a startup specializing in biometric technology, developed groundbreaking new algorithms that reduced the complexity—and thus the system cost—of scanning and matching fingerprints. They asked Mindtribe to help create a proof of concept device that incorporated their technology to secure the contents of a flash drive using fingerprints.

I created a proof of concept USB flash drive using an Atmel AT91FR4042 microcontroller and a Cypress SX2 high speed USB device interface. This built upon our previous work for Biogy’s Biometric Lock Module. I wrote firmware to make the hardware appear as a USB mass storage device whose contents are readable only when the correct fingerprint is scanned.

USB Video Camera Driver (2004)

An optics company built a new type of camera for aerospace and medical applications. They contacted Mindtribe to help them create a Windows device driver for the camera.

I developed a Windows WDM stream-class video capture driver for Windows XP and accompanying DirectX plug-ins/filters to make the camera easy to use with DirectX applications. I also developed a command-line program to control camera parameters.

During this project, I discovered that the WDM stream-class base driver had an undocumented “feature” that caused the video pipeline to stall if the pixel at 0,0 was true black (RGB value of 0). I diagnosed this by debugging into the compiled base driver from Windows and reverse engineering it from assembly. I’m still not certain why this occurred nor why it didn’t seem to impact other cameras. I can only surmise that most cameras never produce true black due to their noise floor. The camera for which I was developing did produce true black at pixel 0,0 due to an in-camera graphical overlay feature that happened to put a black square in that corner.

Contour Follower for Water Jet Cutter (2004)

   

Flow International, the inventor of and world leader in waterjet cutting solutions engaged Mindtribe to explore sensor technologies that would allow them to cut contoured materials. Like a laser, the water jet is focused and needs to remain at a constant distance from the material it is cutting. The ability to cut contoured material requires that the cutting machine can track the distance from the jet to the material as it cuts. The harsh environment and low-visibility in a waterjet cutter makes this a difficult problem. Mindtribe built a sensor head prototype that contacts the material and tracks distance using hall effect sensors.

I led the firmware effort for this design. The firmware runs on a Motorola (subsequently Freescale, now NXP) MC68HC908GR16 microcontroller. It samples and linearizes data from the Hall Effect sensors, applies calibration constants, and sends the sensor data to the waterjet cutter via RS485.

Programmable Delay Line (2004)

     

Colby Instruments, the industry leader in high-precision programmable delay line instruments, needed to upgrade their design from an Intel 8085 microcontroller board that had been obsoleted and was no longer available. They asked Mindtribe to replace the controller with a Rabbit 2000-based module.

I supported the electrical redesign and led the firmware development effort. The delay line uses a stepper motor to adjust the length of an electromechanical “trombone” that physically alters the length of a signal path to precisely delay a signal. We were able to use the old firmware as a reference, but we built all new firmware in C for the new controller module. This included stepper motor drivers, an RS232-based user interfaces, and a GPIB interface. We also added a LAN interface that supports the National Instruments VISA protocol for instrument communication with LabVIEW.

Biometric Lock Module (2004)

Biogy, a startup specializing in biometric technology, developed groundbreaking new algorithms that reduced the complexity—and thus the system cost—of scanning and matching fingerprints. They asked Mindtribe to help prepare their algorithms for production and to produce a hardware reference design that would showcase their technology. This design was to be a lock that could be opened or closed with the correct fingerprint.

I worked closely with the mathematician who developed the fingerprint algorithms to port them to embedded C code that runs on an Atmel AT91FR4042 microcontroller and acquires fingerprints via an Atmel AT77C101B thermal fingerprint sensor. To make theft of their intellectual property more difficult, Biogy wished to keep their firmware encrypted with AES in the microcontroller’s flash. The fingerprint algorithms operate in several discrete and independent phases, so I divided the code up by phase, linked each into the same location in RAM using overlays, and wrote an elf encryption tool to encrypt each phase’s compiled code. I built a runtime loader that would load the current phase’s encrypted code from flash into RAM, decrypt it, perform the phase’s function, and then zero RAM before proceeding to the next phase. Clearly this scheme isn’t foolproof, but it increases the effort required to steal and reverse engineer the intellectual property.

I later built upon this work to create a Biometric Flash Drive proof of concept for Biogy.

Noise Canceling Headset (2004)

   

Jawbone engaged Mindtribe to develop their first noise-canceling headsets. Before creating their iconic Bluetooth headsets, Jawbone showcased their noise-canceling technology via wired headsets. These connected to Nokia phones as smart accessories that—in addition to allowing users to hear each other clearly—could answer calls, change volume, etc.

I worked with Jawbone to reverse engineer the Nokia accessory protocols and write firmware for a Microchip PIC16F84A to allow the Jawbone headset to appear as a Nokia accessory.

Mindtribe also took Jawbone’s first Bluetooth model through production. Our involvement was largely mechanical production support, but I helped out with firmware architecture consulting and audio device drivers for factory testing.

3D Mouse (2004)

Input device startup, Sandio, had a dream of making 3D computer input cost-effective. Existing market solutions were expensive and complicated. Sandio’s idea was to simplify the problem and provide a set of intuitive 3D manipulation controls using creative additions to a commodity mouse design. By adding three joysticks to a mouse design, Sandio was able to achieve straightforward 3D rotation and translation for minimal additional cost. Mindtribe helped Sandio take their design from concept through production with electrical, mechanical, and firmware support.

I led the firmware development effort and wrote production firmware for a Cypress CY7C63413 microcontroller, which provides both the 3D and the basic USB mouse functionality using an Agilent imaging sensor. On this project, I experienced the frustration of not being able to trust the compiler. The only available compiler for this microcontroller merrily generated invalid code for certain valid (and pedestrian) C constructs. My development cycle involved coding, compiling, and checking the assembly output listing for errors before bothering to try downloading and running the firmware. I don’t recommend this, but I did manage to get the product shipped on time.

Automated Bird Identification System (2004)

An enthusiastic and technologically-minded birder was interested in creating a handheld device that could identify a bird’s species by its song. This was before the age of smartphones and apps, so they contacted Mindtribe to explore the feasibility of creating a dedicated hardware device for this purpose.

I led the platform and software development aspect of this research. We flew to Cornell and MIT to meet with top researchers in this field (yes, there is a bird voice recognition field of study). I researched algorithms and processing requirements to understand the hardware cost and development effort for such a system.

Ultimately, our study concluded that an automated bird identification device would be too expensive for even a niche enthusiast market. And that, given that researchers were not achieving compelling results on desktop computers, the performance of an embedded system would be underwhelming. We delivered a comprehensive body of research along with the crushed dreams.

Video Control and Telestration System (2004)

A video telestration startup had contracted a firm in India to develop a large software application that would run on a hardware appliance and allow users to control and annotate live video. Unfortunately, the startup was encountering severe quality issues with the deliverables from the contractor. They originally reached out to Mindtribe to explore the possibility of having us start over.

I evaluated the situation and the code from the other team. I determined that the codebase was not a lost cause, that it made no economic sense to throw it out and start over, and that the quality problems stemmed largely from a communication issue between our client and their contractor. Our client did not have experience providing technical specifications, so they were giving the development team poor direction. The development team was largely guessing at what needed to be built. This was compounded by the geographic separation of the teams. To correct this, I sat down with the client and worked with them until I had a detailed understanding of their desired functionality and I had clarified points of ambiguity. I translated this into a detailed software specification for the contractor. With this improved mutual understanding of what was being built, the relationship between our client and their contractor improved dramatically.

I do not believe in communicating exclusively through specifications; however, sometimes they are the best tool for aligning stakeholders and teams.

Scent Delivery System (2004)

A startup was working on a novel scent delivery system that could precisely deliver a puff of scent to a targeted location in a room. This was intended for entertainment or advertising applications where one might care to target a single individual instead of filling a room with scent. The startup worked with Mindtribe to diagnose electrical noise in their system and improve the design.

I assisted with the electrical design overhaul and improvements to our client’s Windows demonstration application. Unfortunately, the client had lost the demonstration application’s source code they had previously written. Of course, this occurred days before a major demonstration of their technology to investors and they needed to modify the app for the demonstration. I was able to reverse engineer the application binary and patch it with the desired modifications in time for the demonstration. When things calmed down after the big investor demo, I took a moment to teach our client how to use a source control system to prevent future incidents of this nature.

Picoliter Dispenser for Medical Device Testing (2003)

A medical device startup needed to precisely dispense very tiny droplets of fluid for testing and characterization of their product. They asked Mindtribe to help them drive a piezoelectric dispenser for this purpose.

I created a Windows application that controls a 3NETICS PicoSpot piezoelectric dispensing head over an RS485 link (via an off-the-shelf RS485 to RS232 adapter). The application gave our client fine control over the quantity and location of the dispensed fluid.

Augmented Driving Experience Concept (2003)

The BMW Technology Office in Palo Alto was researching new ways of presenting information to drivers using creative heads up display technologies.

I helped them explore one such concept by creating a streaming video system that projects live video onto physical surfaces with varying curvatures. I wrote a Windows application that uses DirectX technologies to map the video stream onto a virtual 3D object that can be manipulated to compensate for the curvature of the physical projection surface.

Automotive ECU Tuning (2002-2006)

   

Dinan is an automotive tuner that creates high quality, after-market performance modifications for BMWs. Their work is highly-regarded and covered by a comprehensive warranty in cooperation with BMW of North America. Unlike software-only tuners, Dinan focuses on hardware modifications that are supported by tuning the Engine Control Unit’s (ECU) firmware to accommodate the hardware changes.

When Dinan started out in 1979, the founders were able to look at an ECU’s flash dump in a hex editor and spot the control tables that they needed to adjust to tune an engine for a modification. As engines became more complex, looking at the data inside the code was no longer sufficient–they needed to understand the code itself. Dinan asked Mindtribe to help them achieve this understanding.

Using the disassembler, IDA Pro, I helped Dinan reverse engineer the ECU firmware for several BMW models, including ones that used multi-processor ECUs. I picked apart the assembly to gain an understanding of how Dinan would need to patch the firmware for a given hardware modification. In order to upload patched firmware to an ECU, I also had to reverse engineer the handshake that the ECU performed to authenticate an authorized uploader. Depending upon the car model, this varied from being based upon a simple CRC to requiring public-key based cryptographic signatures.

Infrared Remote Control for iPod (2002)

   

Consumer electronics startup, Ten Technology, engaged Mindtribe to create an infrared remote control for the iPod. This allows users to control their music collection from across the room. Ten’s remote control has the honor of being the first “smart” iPod accessory and it pioneered Apple’s official Made for iPod program that launched the market for smart accessories.

At the time, Apple did not yet have an official connected accessory program for the iPod. So, I used an oscilloscope and my knowledge of communications protocols to reverse engineer Apple’s wired remote control. I then helped write the assembly code that receives IR remote commands and translates them into iPod control commands, running on a tiny Microchip PIC12C508A.

Ten Technology demonstrated their remote control prototype to Apple in the hopes of having it sold in Apple Stores. Apple was impressed with the polish and utility of the product. They gave us a tiny text file with the official protocol specification and told Ten that if they could revise the product to support an upcoming iPod model, Apple would not only carry Ten’s product in the Apple Store, but they would mention it at the launch for the new iPod. And thus the Made for iPod program was born. We worked around the clock to revise the product and hit the launch.

In addition to working on the remote control itself, Mindtribe helped Ten through manufacturing and created manufacturing test tools for factory operator use. We built the test tools around the Rabbit 2000 microcontroller and wrote the test firmware in a mixture of C and assembly.

Thermal Testing/Profiling Apparatus for Microprocessors (2002)

AMD worked with Mindtribe to build a precise thermal test platform for processor performance profiling under specific conditions. Mindtribe built a modular electrical system capable of controlling heating elements and sampling multiple thermocouples simultaneously. The system was built around the Rabbit 2000 microcontroller.

I led the development team that created the firmware for this platform. This firmware handles data acquisition, logging, and control. I created a Windows application for downloading logged data and exporting it for offline analysis.

Visual Messaging System (2002)

The BMW Technology Office was working with a specialist to create optics for a heads up display in a racing helmet. They contacted Mindtribe to produce electronics for this helmet concept. The helmet would receive messages from a pit crew via a radio on the car’s CAN bus. The helmet would then display graphics pertaining to the messages through the special optics. Drivers would be able to see both these graphics and the road simultaneously without refocusing their eyes.

To drive the VGA microdisplay attached to the optics, we specified a Pentium based C2i-PR5-266 Single Board Computer (SBC) from PFUSystems. To connect to the CAN bus, we built a custom CAN-to-RS232 bridge using a Microchip PIC16F876 microprocessor and Philips (now NXP) PCA82C250N CAN transceiver. I wrote the firmware for the CAN bridge. I also configured and built the Windows CE BSP for the SBC and created a Windows CE application to display the graphics.

Auto-darkening Visor (2002)

   

The BMW Technology Office asked Mindtribe to help them create an auto-darkening motorcycle helmet visor for racing applications. The idea was to create a helmet with an LCD dimming element that could respond to rapidly changing lighting conditions such as when a rider entered or exited a tunnel.

I wrote firmware for an Analog Devices ADUC812 microcontroller that samples a photocell and very precisely generates a driving waveform for the auto-dimming LCD element. I wrapped this up into a firmware application that allowed the BMW Technology Office to use a set of switches and potentiometers to adjust system parameters during their testing.

Chroma-key Toy Proof of Concept (2001)

A large toy company engaged Mindtribe to prototype a concept for a toy karaoke machine that would connect to a TV and use the toy’s built-in camera to show the user against various, selectable backdrops while the user sang along to music.

I investigated various algorithms for chroma keying and background removal and evaluated their suitability for use in a very low-cost embedded system. I delivered a PC-based demonstration application that uses a webcam to demonstrate chroma keying and other forms of background removal algorithms that I wrote in C with a C++ application wrapper.

Automotive Sensor Data Visualization (2001)

The BMW Technology Office was investigating new ways of presenting Park Distance Controller information. Existing systems used audio tones to inform drivers how close they were to adjacent cars when parking. These tones could compound an already stressful situation.

To get the data from the Park Distance Controller sensors, I wrote a complete protocol stack for BMW’s K-Bus—a low-speed, proprietary bus over which non-critical automotive information is transmitted in BMWs. I then used this data to drive a display that demonstrates the BMW Technology Office’s concept. The system is based upon a Motorola (subsequently Freescale, now NXP) MC68HC912 microcontroller.

Automotive IP Camera (2001)

The BMW Technology Office was exploring the integration of an IP-based LAN inside a car. Cars use several CAN and other proprietary buses, which make it difficult to add new devices to a car’s network. Using a standardized LAN could open up new integration and interoperability possibilities. As a part of their exploration, the BMW Technology Office asked Mindtribe to help develop an IP video camera to stream video over an in-car LAN. For a demonstration, the video was to be streamed simultaneously to a web browser on the car’s head unit and to a laptop outside of the car.

At the time, IP-based video cameras were not commonplace. Though a few did exist, they were too large to integrate into a car. Mindtribe built a tightly integrated IP camera using a new USB imaging sensor from Agilent and a small embedded Linux computer based upon the AXIS Communications ETRAX 100LX processor.

I specified and configured the embedded Linux computer, wrote a Linux device driver for the imaging sensor, created a Linux streaming video service, and wrote a Java web applet to connect to the video service and decode/display the streaming video. I learned that I should not make presumptuous graphical design decisions and that large, multinational corporations have logo usage guidelines that do not include their logo’s use as a spinning loading icon while buffering video. The BMW Technology Office gave me polite feedback and graphical assets to use in the logo’s stead. They received the end result with acclaim.

Handheld Gestural Input Device Concept (2001)

A consumer electronics company wanted to explore the use of gestural navigation in a handheld device. They wanted to see how it would feel if the device’s screen acted more like a “window” to content underneath. That is, instead of scrolling the content with your finger, you moved the device itself in space over content that appeared to remain stationary underneath. To prototype this, we instrumented an HP iPAQ handheld device with gyroscopes and distance sensors. I wrote firmware to process the sensors and send the data to a Windows CE application that allows a user to navigate a section of a map by moving the iPAQ around. The first application version panned and zoomed a bitmap image of the map, which did not allow for enough zoom without looking “fuzzy”. I then tried embedding an Adobe Flash player in the app to use vector-based content; however, this heavyweight component did not yield acceptable performance and felt too sluggish. I then tinkered with bitmap tiling to achieve better resolution within the iPAQ’s limited resources, but ultimately wrote an SVG parser/renderer (none existed for Windows CE at the time) and dramatically improved the experience with lightweight vector graphics support.

This concept proved to be a very fun toy, but we thought it impractical for serious use. Our client concluded similarly.

Smart Power Supply (2001)

     

Cyber Switching engaged Mindtribe to develop their first product: a smart power strip that would allow users to remotely monitor and control appliances.

Cyber Switching originally envisioned their product simply as a way to allow network administrators to reboot frozen routers remotely. As the program progressed and they spoke to customers, it became clear that the product needed to offer more. Luckily, Mindtribe’s agile approach allowed us to keep up with Cyber Switching’s evolving understanding of their market. We worked closely with them to help them understand feasibility and costs. We ultimately arrived at a product that could control appliances, but also monitor their power usage and provide alerts or even shut off power when configurable conditions were encountered.

The product offers three physical interfaces

• RS232 - compatible with legacy IT equipment, this interface provides a complete user interface for monitoring, control, and configuration.
• Telephone - this allows administrators to connect the unit to a phone line. They can then dial into it and control/monitor outlets via audio prompts and DTMF input. I worked with contracted voice talent to prepare the script and audio samples that would fit in our limited flash space.
• LAN - this interface allows the units to participate on networks via a number of protocols: HTTP, SNMP, SNTP, and Telnet

I helped with the electrical design of this product and wrote almost all of the firmware. This product was built around the Rabbit 2000 microcontroller—the most accessible TCP/IP-enabled microcontroller at the time. While the microcontroller vendor provided some of the network protocol libraries, I extended most of them and built the SNMP agent from scratch. I also built a dynamic, embedded web page for the product that allowed for near realtime control/monitoring long before AJAX methods were established for doing such things. The microcontroller wasn’t capable of TLS at the time; however, I extended the HTTP server with an HTTP digest authentication mechanism. This type of HTTP authentication was never widely adopted due to the rise of TLS, but it allowed Cyber Switching to provide a cost-effective authentication method that was more secure than sending the credentials in plaintext. At the time, this was above and beyond the industry standard for security in similar embedded devices and it became a selling point for the product that appealed to IT admins.

My codebase was leveraged to build over sixteen products during Cyber Switching’s formative years.

CNC PCBA Drill Software (1998)

Cal Poly has an amazing class, IME 157, that teaches students about electronic product manufacturing processes by having them build an electronic product themselves. In this class, students use CAD to capture a schematic and lay out a PCB. They then drill and etch the PCB, solder on the components, and build a simple sheet metal enclosure. At the end of the course, each student has created a working product such as a car alarm, strobe light, etc.

The IME 157 lab features a high-speed, Computer Numerical Control (CNC) drill for drilling PCBs. When I took the course, the software for this drill was an ancient command-line program that performed no error checking. If a student produced a drill file with errors, such as holes placed outside of the drillable area (very easy to do accidentally in the CAD package), the software would happily rev the drill table up to full speed and slam it into its end-stops. Repeatedly throughout the quarter—despite numerous warnings from the instructor—students would forget to check their drill files. The drill table would slam, the entire class would jump, and the instructor would run to fix the drill so that work could proceed.

Certain that I could overcome the CNC drill software’s limitations, I volunteered to rewrite it and the instructor gave me access to the drill outside of class hours. Finding little documentation on the ancient Delta Tau brand motion controllers, I reverse-engineered their protocol and wrote a completely new version of the control software in Pascal (my most proficient language at the time). This new software alerts users about invalid drill files, displays a graphical preview of the drill pattern before drilling so that students can visually inspect for alignment/scale problems, and improves overall drill time by optimizing the drill’s motion path and S-curve profiles.

After completing the class myself, I went on to T.A. IME 157 for several quarters, becoming an expert in the OrCAD CAD package for PCBA design and continuing to find ways to improve the curriculum and materials.