Watermarking Technology and Research

Much to my surprise I have received email queries requesting the old watermarking paper that was on the old site be made available again. Without further ado here it is, in all of it’s antiquated glory.

I am also receiving queries for information about watermarking, both video and audio. Frankly, if one wants to pick my brain, I suggest hiring me as a consultant.

That said, I believe some queries are simply engineers looking for resources. I thought I’d outline what I do when trying to ramp up on cutting edge research where the end game is actually innovation in the area. Other engineers might do things differently or better, but this is what has worked for me.

First, I do a Google search, Wikipedia read and I call this the superficial newbies and morons informational hunt. I try to just get almost a marketing feel for the concept. What is it, what is it for, what is the basic technique, where does it go, where it is in R&D currently. I also hunt for code snippets, Matlab scripts and try to play with those scripts, write a few lines of code, just get a technical feel for various approaches. I don’t spend a lot of time doing this, maybe a day tops.

Then, the fun comes in. I run down to a University Research library and utilize the vast databases, journal subscriptions of the Academic system and do a literature search. I also perform a patent search on Google patents and the U.S. Patent and Trademark Office. The tricky thing with patents is to make sure my newly formed vague great grand idea and concept has not already been patented. If it is, well, move on and realize great minds think alike. Yet the patent databases are also a good place to review other ideas and evaluate them. It’s also a great place to see what I do not know and do not understand. If I really cannot comprehend a patent, I then go look up, even in an Academic text what I am missing, what equation, what paragraph, what diagram, to understand the paper or patent I’m attempting to digest.

After a literature search, if I find a gem, I might dig in to that particular paper, but very often I am learning about the technology utilized and mulling ideas and approaches from others. I actually move on from the literature review to finding my own approach versus study someone’s work until I could “Ace the test” on it.

One can even take someone’s idea from a literature search and code it up in Matlab to see how well it works if the brain gears still are not turning. But generally speaking, this is the point I am writing code to learn more about the technology area I’m exploring as well as writing some code to see if my approach and idea is even feasible.

For me, I find writing up some code, drawing a block diagram, even breadboarding a circuit, trying something, is the way I figure out what’s going on. Even if my first efforts are huge fails, I learn something by doing instead of just thinking. I get something on paper, screen, digital bits, somewhere. If I stay stuck in my head, the idea and concept stays there too instead of to prototype.

The above is just a little bit and I hope it helps someone out there. I guess a key is to not be afraid, not be shy. It is our right as engineer explorers to be dumber than a box of rocks and technically clueless periodically. In other words, don’t be afraid of these massively complex engineering research areas. One foot in front of the other, even while banging around in the dark has always gotten me somewhere.

Hey man, turn the light on!

About The New Site

Folks, I just put up this new site.  It is WordPress and chosen because it is responsive design, accessible from anywhere and simply mega easy to style, modify and maintain.  Expect to see more details as I add all sorts of techno babble, skills and projects on the site.


For the moment I am a consultant who works remotely.   I have strong expertise in audio signal processing, compression standards, in particular ISO/IEC MPEG, as well as ITU G.7xx speech codecs.  I also am an optimization expert and live to write cryptic assembly that I actually comment so one can follow along at home. C is my best language with my Kernighan and Ritchie still with me in tatters.   I do have many other languages, including the antiquated Fortran and the obligatory C++.

I have a strong hardware background and have implemented projects on all sorts of embedded architectures. Yes, I do run a popular economics site, based in Drupal 7.x, where I did all of the coding modifications.  The site currently resides on a CentOS 6.x, unmanaged VPS that I configured and maintain.

I build my own systems too.  They will have to pry my Wiha T4 Torx from my cold dead hands, I am that geeky.

Other skills will be added to the site and overall I’m out to be more chatty.  I am a notorious lurker who posts little, yet  knows a lot.  We’re out to change that.

H.264 in WebRTC

Cisco just did a solid and released H.264 into open source for use in WebRTC. Cisco is absorbing the estimated $13 million in licensing fees from MPEG-LA. The reason Cisco did this was to avoid transcoding of H.264 streams from other video clients, a wise choice. The IETF is determining which video compression algorithm will be the baseline for webRTC this week.

Mozilla will incorporate webRTC, including H.264, into the Firefox browser and Chrome already has it. Google is sticking with VP8 and is a major developer and contributor to webRTC. The IETF vote is in Atlanta this week and we’ll update this post with who won out in the video codec selection. We hope both are supported, yet Cisco is right, the last thing one wants to be doing in a real time stream is transcoding H.264 to VP8. It does seem that stand alone proprietary video conferencing and streaming solutions are going the way of the dinosaur, although webRTC as the meteor which killed the beast is yet to be determined.

Real-time communications in the browser utilizing native codecs and javascript sounds too good to be true and that is becuase it is. Underneath the hood are the typical real time streaming issues such as NAT transversal, firewalls, latency, QoS and synchronization, to name just a few. WebRTC latency was reported to be 150ms with a mininimum of 100ms. While that mets ITU QoS, it is pretty much the crappy out of sync video conferencing most of us experience daily.

It appears at the IETF meeting there was no consensus on which video compression algorithm to make mandatory for webRTC.   It also appears both is not an option, so far and Nexus has hardware implementations of VP8 while hardware acceleration for H.264 is available in many chipsets.

On a technical note, I am downloading the source for webRTC, focus on Opus, (Xiph), since supposedly it is not ported, never mind optimized, on many embedded platforms.    I’ll write up what I find in another post.

Tools Which Make Remote Consulting Work

Telecommuting saves the environment, allows those with disabilities to more easily perform tasks,  increases productivity and removes distractions that are common in an office environment.    Yet, many employers still do not take advantage of new technologies which enable working remotely.

Here at Musemagic we utilize Skype, ftp, servers,  remote access, chat clients, and our personal favorite, VMware Workstation.   We can create VMware images with not just the deliverables, but the entire development system and test bench.  One can see exactly every step of development and the results by this method.  All one has to do is pull up the image in their VMware player and hit go.    VMware can convert physical machines into virtual ones and vice versa in minutes and enables remote access to a virtual machine.

For embedded projects where JTAG USB drivers are not compatible with VMware (we’ve yet to see this happen), we can also configure a laptop with all development tools for that particular embedded environment.  Evaluation kits are cheap, custom boards are also easy to transport and one can mirror development as well as burn the flash ROM with the latest code drops in minutes.  Of course being able to set up systems, solder daughter boards, design circuitry and other skills one must have to bootstrap a project are a given.

We also can set up Mantis, a bug database, and a code repository of your choice.    With today’s video conferencing and  on-line meeting tools, literally we can be present on your desktop anytime you wish.          This is just a short list of technologies which make remote consulting of all types work.

Our Skype ID is musemagic.  Give us a buzz, even for casual queries.

HEVC Video

The ISO/IEC JTC 1/SC 29/WG 11, a.k.a. MPEG video – ITU video, a.k.a. JCT-VC,  has issued a committee draft of H.265, or HEVC.    HEVC stands for High Efficiency Video Coding and is the next generation video coding standard.     The algorithm’s main goal is to achieve the same video quality at half the bitrates of H.264.     More resolutions are also offered.     The final standardization approval date is scheduled for January 2013, with a standardization draft due in July 2012.

The Fraunhofer site is maintaining the source and some documentation.

We’re following HEVC as it develops.     Here is a pdf of the  HEVC requirements document, w11872.      The requirements document does specify video conferencing.      This implies a low algorithmic delay and overall  low latency HEVC profile, at minimum.     The fundamental trade off of the standard is increased computational complexity to achieve higher compression efficiency.     Below is the HEVC application list from the MPEG  requirements document.

• Home cinema
• Digital Cinema
• Surveillance
• Broadcast
• Realtime communications, video chat, video conferencing
• Mobile streaming, broadcast and communications
• Storage and playback of video
• Camcorders
• Video On Demand
• Internet streaming, download and play
• Remote video presentation and remote computer graphics display
• 3D video
• Telepresence
• Content production and distribution
• Medical imaging

3D Video

There have been recent queries on 3D video technologies.    Currently the MPEG standards body has created an amendment to the H.264 (AVC) video standard, approved June 2009, called Multiview Video Coding.

The amendment covers:

  • Stereoscopy
  • Multi-view
  • Free viewpoint

The MPEG H.264/AVC MVC amendment is backwards compatible to older technologies & MPEG-4/H.264 implementations that do not support 3D display capabilities.

Nokia Research is hosting the reference source code and corresponding standards drafts.

I am currently working with the above references, Fedora 13 OS platform. To date, the open source x264 project, part of VideoLan, emulating H.264 streams, does not support the MVC amendment.

Remote Consulting

There is much misunderstanding of remote consulting.    Many companies have been burned, sometimes badly, by less than ethical consultants who promise the world and deliver nothing.  They may have offshore outsourced a large contract, only two years later to be out millions of dollars and have no results at all.  They might have received incomprehensible spaghetti code or deliverables which literally are a black box to them and no one available to support that project after the fact.

Firstly, I am in the United States, working with Companies under the jurisdiction of Oregon state law and U.S. federal law.  One doesn’t need to worry about international law or lack thereof to ensure they are getting what they paid for.

Secondly, Contractual law enables fairly narrowly defined business relationships if one wishes to write them into a contract.  For example, I write into contracts support clauses as well as intellectual property defense clauses.  This guarantees after the project is complete, if there is a support issue, a question on implementation or God help us all, an intellectual property challenge of some sort, as long as payment is received, I am obligated to support the work.    These clauses are actually superior to an employee, for once that person is gone, there is no getting her/him back without some sort of renegotiation.

Many believe that team building can only happen in an office, or in a physical space.   Remote consulting is really more of a paradigm shift.   One can see from the massive build up on the open source community as well as social networking, the blogosphere, that advances in technology, when utilized, can structure a well organized and managed project that is almost superior to the traditional white board meetings on site.   The reason for this is the electronic paper trail remote communications generate.  Every instant message has a logging system.  Online meeting software literally enables users to control the other’s computer remotely.  VoIP systems have call logging (recording) capability.  For those who want to remember what was said, these technologies  are literally instant note takers and assist greatly in focusing meeting topics, clarifying requirements and detailing project execution.  With today’s technology, one can have an online brain storming session or white board meeting which is in many ways superior. The software itself logs everything, from chicken scratch to the seed of the next killer app for you.  Nothing is lost or forgotten by software, unlike my brain without a cup of coffee.

Remote access to CVS repositories, the advances in Subversion for software control, enable ease of project management better than peering over the Cube wall and asking if that class or function is integrated yet.

Sometimes, in spite of all of this, on site might be necessary.  This can be especially true in large embedded, firmware projects or large custom boards with integration issues.    I will come on site as needed in these cases.  One can be quite surprised at just how much can be achieved offsite even with large integration issues and possible software module resource conflicts and other integration bugs though.  I have been remote consulting for years and when the consultant is responsible, professional as well as the client, it is astounding what one can achieve remotely.

There are other sorts of consulting that require travel, such as to standards bodies.  My general rule of thumb is if the work itself truly requires on site or travel, I will do what it takes to get the job done in these cases.

Let’s Talk Consulting Rates

Anyone considering using a consultant needs to know their total outlay of expenses.   There are also different methods of payment for consultants.   I am a sole proprietor, for tax purposes.   This means all payments are 1099-misc., and you are contracting with a small business. It also means you are not responsible for FICA, additional social security, workman’s compensation, insurance, retirement, office space, utilities, recruitment fees or 3rd party add on fees.

To compare a 1099-misc. consulting rate to a full time employee salary, a good general rule of thumb is to take what the base salary would be and divide by 1000.   Let’s say you believe a skill set is market value @ $120k/yr, FTE.   Then, the corresponding 1099-misc consulting rate would be $120/hr.

I give different rates for different projects, skill sets and time lines.   Bear in mind, Consultants must be their own sales, marketing , accounting and tax advisers.    So for each new client, there is a period of negotiation and discussion where the consultant is not actually being compensated by the potential client.   All of these costs are small business expenses and should give a little insight as to why a consultant is not equivalent to a permanent full-time employee.    The reality, for a typical corporation in terms of total business expenses, the consultant can be less expensive or equivalent.

For those used to thinking in salary terms, these differences are critical to realize.   While one may perceive a consultant is highly compensated, in reality they are earning, in net income terms, probably the same as a full time employee or less.   More importantly your total expenditure outlays are the same or less.

Here is a employee true cost calculator which can assist in seeing these hidden expenses for those who think in terms of salary and not CFO/COO overall business costs.