Adobe Community Moderator
Posted by Jay Charles | Filed under General Nonsense
Last week, I was invited to become an Adobe Community Moderator… an invitation I’ve gladly accepted. I spend a lot of time contributing to support forums [the Adobe FMS forum in particular], and while I don’t do it for recognition, it’s really nice to know that my contributions are valued.
As a Community Moderator, I’ll help with managing comments posted to the Adobe LiveDocs, and contribute notes, comments, etc. where I can.
FMS Connections Failing On Mac OS 10.5? It’s not your fault.
Posted by Jay Charles | Filed under Flash Platform
A client of mine recently came to me with reports of Mac users having problems connecting to their streaming video application. From the user’s view, the video simply wasn’t playing, creating suspicion that there was an FMS code or Flashplayer bug at play.
Although I’ve become [for the most part] a die hard Mac hater, I keep a couple of Macs in the office for testing applications (or perhaps more to the point, seeing what doesn’t work on Mac, or makes the browser crash on Mac, or what gives me that stupid spining color wheel on Mac, or…). One runs OS 10.4 (Tiger), the other runs OS 10.5(Leopard)
Since I knew for sure that my client’s FMS application was well coded, and I knew that it works flawlessly on Windows, I came into the investigation with reasonable confidence that the problem has to be specific to the Mac. My OS 10.4 machine connected without issue, so it was clear that the problem is OS 10.5 specific
While testing on OS 10.5, I found that I was able to connect to the application, but it was taking an unreasonable period of time for the connection to be established… upwards of 25 seconds. I further found that refreshing the page (after connecting) resulted in a successful connection almost immediately.
That result made DNS issues my primary suspect. After clearing out my dns cache, I started a tcpdump session to see what was going on. It turns out that OS 10.5 is first attempting an IPv6 SRV lookup, rather than an IPv4 A record lookup. In fact, the Mac was trying several SRV lookups before finally falling back to an A record lookup.
Unfortunately, if your router, your DNS, or the host DNS does not support IPv6, you have to sit there waiting for all of those SRV lookups to fail before you finally get your A record lookup. That’s not such a big deal when it comes to HTTP requests, but it can be catastrophic for FMS applications. When the Flashplayer initiates a connection, a timeout is set. If the connection isn’t successful or rejected before that time runs out, the Flashplayer throws a failure event. Since waiting for those SRV lookups to fail can take a while, there are situations where the netconnection.connect() will time out before the DNS lookup is completed. Although this isn’t a flashplayer or FMs bug, the problem makes it appear [to some OS 10.5 users] that your application is broken.
There are two ways I can think of working around this problem:
1. Use IP addresses for your FMS servers instead of domains. Of course, this means that you can’t access a named Vhost… all of your junk has to run from the _defaultVhost_ .
2. Rework your client side netconnection’s status event handler to retry the connection after a failure. This sucks to the Nth degree, as it means you have to delay failure handling even for genuine failures, so your users get to wait even longer to be told the server isn’t available.
Personally, I hate both of those approaches, as it means limiting your deployment options or degrading your application to serve the poor design of a specific version of a specific vendor’s operating system.
Some might argue that OS 10.5 is actually doing the “right” thing by embracing modern standards. I suppose I can see the point way off in the distance there (blurry… but I see it). The problem there is that [most of] the rest of the world is not running on IPv6 and SRV yet (how many hosts/sites/service providers do you know to be using IPv6?), so really, this is a case of Apple engineers shooting their customers in the foot, and making life generally aggrivating for developers like me.
As best I can tell, there is no OS level configuration setting for this, so if there’s anywhere in the lookup process that IPv6 isn’t supported you’re stuck waiting each time a DNS lookup is done, making this item #26580 on my list of reasons to hate Mac OS. Don’t believe the hype… the pretty box is a lie (although it makes for a nice doorstop).
Are RTMPE and SWF Verification False Security?
Posted by Jay Charles | Filed under Flash Platform
A discussion today on the Flash Media Server List and a post on Stefan’s blog brought to my attention a DMCA takedown order Adobe issued to SourceForge, requesting that SourceForge remove the Rtmpdump project. The details about Rtmpdump and the takedown order are pretty well covered in the links… you can read in there if you want the nitty gritty on the story.
In most cases, this is the point where I’d fly into a rant about how open source fundamentalists need to realize that not everything is theirs… but that’s not the point of today’s post. If you’re in the mood for that part of the discussion, it’s pretty well handled at Slashdot.
What concerns me today is the information posted here and analysis posted here (written by someone who is clearly biased, but seemingly knowledgeable) suggests that neither RTMPE nor SWF Verification are not as secure as one might think.
The report suggests that all of the information required to compromise RTMPE and SWF Verification can be obtained from the publicly available .swf file alone. This is something I’ve always suspected of SWF Verification, but the idea that the same may be true of RTMPE is a bit disheartening.
It will be interesting to see how Adobe reacts to this. In the meantime, I’m not making any RTMPE recommendations (and feeling a bit relieved that I’ve never implemented RTMPE to satisfy critical security requirements on any past project).
Singing the Praises of Microsoft?!
Posted by Jay Charles | Filed under Rants and Not-So-Rants
It’s not all that often I have anything great to say about Microsoft… but today brought a particularly good experience. Given that I’m pretty quick to jump on the soap box and start complaining when something is wrong, it seems only fair to sing praises when something is right.
Today, I was forced to install Service Pack 1 to the Vista machine on which I do most of my development work. I typically don’t install service packs unless it’s absolutely necessary (SP installs usually = downtime fixing things), but since SP1 is a requirement for Adobe CS4 (I’ll complain about that later), installing the service pack became necessary evil.
My first attempt at installing was the failure I expected… Windows made it to stage 3 of the installation, and then stalled at about 75%. Life was bad, and I had visions of never being able to use that machine again (I’m being dramatic there… one can always reformat and start over).
I went to the Microsoft site looking for help, and after going through all of the KB articles and not finding the fix, I submitted a support request. Quite honestly, I was expecting to find that support would be an endless chain of emails with people who didn’t understand the problem. To my surprise (and to my pleasure), I was immediately directed into a chat session with a real, live support person. Even more to my surprise, the support tech initiated an “Easy Care” session, jumped on to my desktop, and fixed the problem for me. Really… no fooling… the guy just fixed it, and his manager called me half an hour later to make sure everything was working.
I am simply floored by the experience. Whoever is running the show at MS support is really on their game today.
I thought I’d never say this, but it appears that Adobe could learn a few things from Microsoft. Seems a bit backward when I can get immediate, one-to-one support with a well qualified tech for a $300 operating system, but I can’t even get an email response to support requests for $4500 server software.
FMS and its “built-in” HTTP support: First Impressions
Posted by Jay Charles | Filed under Flash Platform
Flash Media Server version 3.5 sports some new HTTP features that, on the surface, are really sort of exciting. The idea that a single deployment might support Flash video over both RTMP and HTTP makes my inner geek smile.
FMS 3.5 came out a few months ago, but due to workload and lifeload, I just hadn’t had time to test out the HTTP goodies. Last week, one of my clients came to me with a challenge that required diving headlong into FMS’s HTTP support.
The trouble my client was having was that he couldn’t find how to enable HTTP support for FMS Vhosts. After a short investigation I found the cause of the trouble… there is no built-in support for consuming HTTP resources from a Vhost.
Out of the box HTTP support in FMS works like this:
- Upon installation, FMS installs Apache 2.2, configured to bind to port 8134
- A “webroot” folder is created in the FMS installation directory
- FMS handles all requests for both RTMP and HTTP
- When FMS encounters an HTTP request, it proxies that request to port 8134
- Apache goes looking for a file in the default webroot directory
… that’s it. Once FMS hands off to Apache, FMS is out of the picture. It’s then up to Apache to do the rest of the job.
Now, if your FMS/Apache deployment uses all of the default configs, and if you don’t need to support HTTP for multiple Vhosts or support for serving files over HTTP from the same location from which you serve RTMP (a la HTTP’ing files from your FMS Vhost directory structure), this arrangement works quite nicely. You dump your files into the preconfigured webroot (as well as into the director you’ll use for serving via RTMP), and you’re done.
In my client’s case (and in the case of just about everyone else I’ve built FMS apps for), multiple Vhosts and/or delivering media files from application directories (without duplicate files elsewhere) are requirements… and in this particular case, the only option was to enable Vhost support on the Apache FMS installed.
I like to think of myself as a fairly smart guy, and I’m reasonably quick when it comes to picking up on new things… but administering Apache is an area of broad scope. After two days of doc reading, forum searching, and some help from folks on the FMS Listserv , I found my way through to what needs to be done. The first half of the job involves setting up a corresponding Vhost on Apache for each FMS Vhost (or creating a wildcard Vhost on Apache… that’s the route I took) using mod_vhost_alias, and the second half is to create rewrite rules to translate HTTP URL’s to the application directory structure within the FMS Adaptor, using mod_rewite.
Right now, anyone who has previous experience with Apache is thinking something along the lines of “well duh Jay, I could have told you that”, but having never worked with Apache in the past, it was a challenge to get it done. That’s not a complaint, but it does raise a complaint…
Now that FMS is being advertised as having “HTTP delivery support” included, many of my clients are going to expect me to get it working regardless of the app structure (after all, Apache is now “part of FMS” so to speak). Non technical people (you know… the ones that are usually writing the $4500 check for FMS) are apt to assume that since the marketing speak used on the FMS product page simply says “HTTP delivery support” it means HTTP support is native to FMS and FMS can serve up all of your media over both HTTP and RTMP. That’s not really the case. What FMS can do is proxy requests to an httpd server… but unless you’re using the default httpd configuration and putting your files into the default webroot as set up by FMS, you’re on your own to configure Apache to handle the requests as you need them to.
My personal take on this is as follows; If Adobe is going to market FMS as supporting HTTP delivery, there ought to be some tighter integration between FMS and the httpd server. Of course, there’s no way the FMS engineers can provide automatic support in all situations, but it seems to me that the included httpd server (in this case, Apache) could be/should be pre-configured to handle what FMS developers have come to know as common configurations (i.e the typical application/streams/instance/file.xxx structure FMS creates when recording streams). Even better would be extending FMS to support making Apache configuration changes a task that would be handled through the FMS admin console (that would actually be pretty sweet… a bit of GUI that lets an FMS admin associate directories in the FMS adaptor structure with HTTP URL’s).
Barring that, I really think the point about HTTP support should be removed from the FMS product page, or at very least qualified with a description of what can and can’t be done before you need to get your hands dirty with Apache configuration changes.
AMF3 Server side NetConnections + admin service data = crash?
Posted by Jay Charles | Filed under Flash Platform
I’ve been testing a lot of my code against FMS 3.5 over the past couple of weeks, and I’ve hit an interesting snag when dealing with data returned from the FMS admin service API methods.
The scenario is as follows:
There are two FMS applications…Application “A” polls the admin services of a cluster of FMS servers, to keep track of instances and instance stats. Application “B” connects to Application A, and requests the collected admin service data on an interval (once a second).
Application A works fine on FMS 3.5, but application B was choking on the data and crashing after 20 seconds or so. I spent a few days deconstructing the application to find the problem… and when I was down to a single hair left on my head, I stumbled across the solution… setting the object encoding of the netconnection to AMF0 stopped the crashing.
Now, truth be told, I don’t know why this is. I’ve passed the info along to someone on the FMS engineering team, and hopefully whatever the bug is will be squashed in the next update. But, for the short term, if your FMS apps are crashing on 3.5, and you’re using admin service data, try setting your netconnection’s object encoding to AMF0.
Replay Media Catcher and the nc.verifyClient method
Posted by Jay Charles | Filed under Flash Platform
I recently helped someone (on the Adobe FMS support forum) who was having a problem with Replay Media Catcher being able to steal video content from what was assumed to be a protected FMS application. I’ve decided this one is worthy of a blog post, as it may help some others who find themselves same situation.
Not long ago, Kevin Towes posted a whitepaper outlining methods to use for verifying clients connecting to Flash Media Server. A focus of this whitepaper is how to prevent Replay Media Catcher and other RTMP “rippers” from consuming video from an FMS application. One of the methods described is to use a remote method invocation, whereby the server calls out the client with a random string or series of numbers, and the client returns that string to the server (see page 25 of the whitepaper for the sample code).
It seems that the developers of Replay Media Catcher read that whitepaper, came to the conclusion that lots of people would just copy and paste the sample code, and decided it would be in their interests to add a method named “verifyClient” to Replay. If you use the sample code from the whitepaper as-is, Replay will be able to defeat your protection.
The fix is a simple one… don’t use verifyClient as the method name. Just change it to something else, like nc.replayCantStealMyVideo (but don’t use that either… the Replay developers might be reading this right now and adding it to their code).
Link to the whitepaper: http://www.adobe.com/devnet/flashmediaserver/articles/protecting_video_fms.pdf
Adobe Max 2008 NA Videos on Adobe TV
Posted by Jay Charles | Filed under Flash Platform
I just read a post on Ted Patrick’s blog informing that, starting during the first week of December the Adobe Max team will begin releasing some 290 hours of video recorded at Max 2008 NA on Adobe TV.
There aren’t words to describe how much I appreciate this. Having not been able to make it to San Fransisco this year, I’ve been feeling like I’m going to be three steps behind the game for the next year. Big thanks (or ‘props’, if you feel like being a hipster about it) to the Max team for making this happen, and big thanks to Adobe for putting it out there for free.
Testing 1…2..3
Posted by Jay Charles | Filed under Flash Platform
I’ve posted a demo of a client’s application for open testing. If you have some time to play, please jump in and give it a go. If you run into any unexpected behavior or (if you’re running a debug flashplayer) runtime exceptions, please let me know about it so I can squash the bugs.
Spam City
Posted by Jay Charles | Filed under General Nonsense
Sometimes, something can be so stupid that I have to just laugh at the situation. During my last migration, I took down the spam filter on the old server (the one where my old domain lives) to transfer the license to the new server. It’s been a couple of weeks since I checked the email on the old domain (the screen shot below has not been Photoshopped)…
