Category: Software Engineering

Coding, Python, web development, architecture, and deployment

  • Progress on New Game

    So the new game is going to be called Tycho City. It’s based on building a lunar base and battling other bases for the moon’s valuable resources. There’s a mix of two styles of gameplay the first is the simulation of the moon base with building and upgrading mines and energy collectors. The second is a defence game where you attack other bases by landing your troops/robots and taking on their defences.

    Unlike my previous games this time I’m taking some good advice and modeling other successful games. Rather than try to invent something brand new I am going to try to put my spin on a popular and profitable game concept.

    One thing about this type of game is just how much data there is to manage.

    This game will be heavily dependant on web servers to deliver almost everything. All of the costs, upgrade paths, graphics, assets, music, rules, build times, and store menus will be defined and controlled from my web backend. It will allow us to dynamically update and tweak the game play to optimize things after it’s released and keep it fresh with new content. But it’s a lot of Data. And that’s just on the static side of things. The server will also keep and save the game state of every player’s game in real-time.

    Organizing all of this information in a way that is going to be easy to understand and work with is not easy. But it’s a fun challenge.

    This game is going to take several more months to finish. But it’s going to be my best chance of getting a major hit.

  • iAd Download Script

    Unfortunately Apple doesn’t have a nice easy API for downloading your revenue numbers from iAds. The only way around that right now is screen scraping the information, or manually downloading the reports.

    Thanks to the awesome python requests library I was able to put something together in just 50 lines of code.

    It’s a function you can integrate with your project to download daily iAd report files.

    It is also simple enough that it should be relatively robust against future website changes by Apple. (hopefully)

    If you decide to use this I would recommend storing your account’s publisherID to avoid having to get it each time which will speed it up.

    (this code is also on github: https://github.com/mfwarren/iAdDownload)

    import requests
    import re, os, datetime
    
    url_base = 'https://itunesconnect.apple.com%s'
    signin_url = url_base % '/WebObjects/iTunesConnect.woa'
    download_csv_url = 'https://iad.apple.com/itcportal/generatecsv'
    
    def getPublisherID(session):
        """
        publisher ID is a number that is buried deep... lets hope
        this code doesn't break often
        """
        headers = {'Content-type': 'text/x-gwt-rpc; charset=UTF-8'}
        iad_service_url= 'https://iad.apple.com/itcportal/service/startup?requestId=GUEST_1331628761939_0__1331628761950__null'
        body = '5|0|4|https://iad.apple.com/itcportal/itcportal/|E8DB97D87973D76A7C9096DCF8A83BB5|com.qwapi.portal.itc.client.rpc.IITCStartupService|getStartupData|1|2|3|4|0|'
        r = session.post(iad_service_url, data=body, headers=headers)
        endpos = r.text.find('com.qwapi.portal.client.rpc.dto.UserDTO')
        endpos = r.text.find('"', endpos-4)
        startpos = r.text.rfind('"', endpos-20, endpos-4)
        pubID = r.text[startpos+1:endpos]
        if not int(pubID):
            raise Exception("Failed to get publisher ID")
        return pubID
    
    
    def downloadiAdFiles(appleId, password, publisherId=None, outputDirectory='.', daysToDownload=14, outputFormat='iAd_D_%m-%d-%Y.txt'):
    
        session = requests.session()  #session used to maintain cookies etc.
    
        #get signin page to find url for posting signin credentials
        r = session.get(signin_url)
        match = re.search('" action="(.*)"', r.text)
    
        #login to ITC
        params = {'theAccountName':appleId,
                'theAccountPW':password,
                '1.Continue.x':'10',
                '1.Continue.y':'10',
                'theAuxValue':''}
        r = session.post(url_base % match.group(1), params=params, headers={'Content-Length':'0'})
        r = session.get('https://iad.apple.com/itcportal')
    
        if publisherId is None:
            publisherId = getPublisherID(session)
    
        reportDates = [datetime.date.today() - datetime.timedelta(i + 1 ) for i in range(daysToDownload)]
    
        filenames = []
        for downloadReportDate in reportDates:
            filename = os.path.join(outputDirectory, downloadReportDate.strftime(outputFormat))
            if (os.path.exists(filename)):
                continue
    
            dateString = downloadReportDate.strftime('%m/%d/%Y')
            params = {'pageName': 'app_homepage',
                'dashboardType':'publisher',
                'publisherId':publisherId,
                'dateRange':'customDateRange',
                'searchTerms':'Search Apps',
                'adminFlag':'false',
                'fromDate':dateString,
                'toDate':dateString,
                'dataType':'byName'
            }
            r = session.get(download_csv_url, params=params)
            if r.status_code != 200:
                raise Exception("Script failed to dowload - check login credentials & publisher ID")
    
            with open(filename, 'wb') as f:
                f.write(r.content)
            filenames.extend([filename])
    
        return filenames
    
    
  • Winter Bike Riding

    So I decided that rather than paying for driving/parking to get to work this winter I am going to bike.

    I picked up a cheap bike from canadian tire last night that should be able to survive the -30 degree temperatures outside and should be sturdy enough to not break down on the 6KM ride each way to the office.

    It’s a fixed gear urban bike with disc brakes. The rims are pretty beefy and it should hold up to a lot of pounding.

     

    It’ll be nice to be back on a casual bike again.  I can start brushing up on all the tricks I learned growing up.

    This is how I plan on getting to work:

     

  • Loading iOS Assets From The Web

    One of the big challenges with making a big iPhone game is that to do the graphics justice at the caliber expected by gamers these days is hard. There’s just too much data to package the iPhone/Retina iPhone/4 inch iPhone/iPad/Retina iPad assets all together in one bundle.

    So for my latest game I’m kicking things up a notch and adding a fairly complex loading screen which will connect to my web server API and download much of the content of the game from there.

    There are of course some downsides to doing this. It requires that my web servers have great uptime, and are performant enough to support all the players connecting. The added complexity means more code to maintain and debug.

    However it gives a ton of options for tweaking things on players and improving the game without deploying new versions it through the App Store. It opens the door for the opportunity to split test game play options to find what values create more engagement.

    If the game code is flexible enough I will be able to download new assets, change prices, run sales and promotions, and add more complex multi-player interaction.

    The mobile gaming market is becoming more sophisticated very quickly and the quality required to stand out in the crowd is amazingly high. Developing something that is just OK will be a waste of your time and flop. Being an indie game developer for mobile is now a difficult undertaking.

    When the code for the loading screen is done I may make it available as a component for other developers to use in their projects.

  • Creating a Resource Management Game on iOS

    Over the last week I’ve been trying to reverse engineer some of the hit games on iOS like DragonVale, Pocket Planes, Tiny Tower, Simpsons Tap Out and Paradise Cove. These have been historically some of the most profitable games on the App Store. They are complex resource management type games and I want to create one myself.

    The first thing that you notice is that unlike most other games on iOS these ones have a loading screen which can actually take a while, and they require an internet connection. I actually made a couple pages of notes just on the loading screen to understand what it needs to do.

    One thing that these games seem to have a lot of problems with is the economy. None of them get it quite right. Though I think Tiny Tower is perhaps the best of the bunch. Paradise Cove is frustratingly slow (perhaps intentionally) where the money you make is rarely enough to build the next thing. DragonVale on the other hand makes it easy to get rich and it becomes hard to spend all the money you make fast enough.

    Developing a good economy should (at least in theory) be a lot like the real economy. The more money that is created in the economy the more “inflationary” pressure there should be on prices. A game should allow for some interesting modeling of these effects and would make for an interesting platform to test some of those ECON 101 theories.

    The technical aspects of creating a big game like this are fairly intense. There are some big server requirements to store the game state for every player who plays and APIs to keep the device in sync with the server. This is required to allow for playing the same environment across multiple devices, allowing the player to view/interact with their friends game, preventing cheating, and it gives me, the developer, the ability to bug fix and support players if problems come up.

    The other big thing about these games is that they won’t work if there isn’t some simple side games within them. Many of these games have simple games of chance or other game mechanics other than just the resource management and expansion to play with. DragonVale has the collection of cross-breeds, Paradise Cove has the mermaid and pirate fights, The Simpsons has killing the zombies. These extra game mechanics can make the game significantly more engaging.

    So how much work is it to create one of these games – even a very simple one? A lot. Even starting with a base of code from my other games I’m estimating at least 3 months of full time work to finish one. Long enough that there’s a good chance of failure. No risk no reward!

  • Getting ready for Winter

    Winter seems to have struck early here in Calgary.

    It’s left me with a bit of a conundrum for getting to work.  All of the transit options in this city suck including driving your own car.

    • The buses are never on time, and always delayed or late
    • The C-Train doesn’t get me to the office, and gets hung up a lot in the cold and doesn’t run often enough
    • The roads don’t get cleared or salted, and it gets to -40 degrees so biking is both dangerous and frigid
    • The side walks also are icy, not cleaned so walking is slow, dangerous and cold.
    • Parking is stupid expensive in this city, and traffic sucks
    • Innovative car sharing option car2go trades the parking concern with finding an available car within walking distance

    Luckily I only have to be in the office 2 days a week.

  • A New Game Project

    It’s been a while since I’ve worked on an iOS game.  After the disappointing sales from Zombie Tap Out I figured that the game section of the App Store was getting a little to difficult to compete in.

    I’m finally ready now to get back in the game and start working on something new.  The idea for this next project is to actually make a solid business investment and try to create the best possible game I can make.  To get it working it’s going to be a team effort and will require bringing in some talent to do the graphics work and making sure the game is fully tested before releasing it.

    This game is going to be one of the best ones that I’ve been playing myself and which has been successful in the app store.  The resource management style gameplay of games like Farmville are strangely addictive and is what I’m going to attempt to build.

    These games are deceptively difficult to create.  There are so many art assets required that it’s not possible to package them in the app and so they need to be provided from a webserver.  Being able to optimize the gameplay without submitting a new game to Apple requires many of the game rules to be up-datable on the go.  And preventing cheating requires saving the game state for every player and keeping the game in sync between the server and all the users potential different devices – resolving conflicts when they come up.

    going to be a real challenge,  but also lots of fun, and lots of potential to be a big hit.

  • ABT: Always Be Testing

    Sales teams often have the letters ABC plastered somewhere next to their weekly leader boards and top salesman photo. It stands for Always Be Closing and it’s a motto that keeps sales people laser focused on their goals. Marketing teams should have a similar motto: ABT – Always be testing.

    Lately I’ve had the opportunity to do a lot more testing on the performance of webpages and apps. One thing always holds true. Nobody knows what’s best – which is why it’s always important to test things.

    This point cannot be more clear. If your head of marketing, lead designer, or CEO could peer into the minds of potential customers and knew the most effective way to sell to them, then they would probably be living on a beach somewhere. The fact that they keep coming into work by car instead of helicopter is enough evidence to show that they don’t have all the answers.

    One of the things that gets under my skin about marketing is the focus on branding and brand related marketing. Reading a book about branding is like taking a walk back in time to the 1980s. It’s difficult to find good recent examples of successful brand marketing because more and more companies are discovering that it’s simply not effective. With more technology to play with it has become easier to track the performance of marketing efforts. TV ads can be tested in different regions. Web banners can be split tested daily to find the most effective ad copy.

    You know what tends to happen when tested and accountable marketing comes into play? The clever catch phrases disappear, gimmicky websites get toned down and focused, and communication becomes more direct.

    I recently heard a pitch from a brand marketing consultant who tried hard to impress upon us the importance of a good tagline as the basis for all marketing. It was not very good advise. One of the examples he used was Avis and their famous “We try harder” tagline. It was claimed to be a tagline that helped grow the business a lot. Question is if taglines are that important why aren’t they presented up front in current marketing.

    “We Try Harder” shows up as a footnote on Avis’ current website – barely noticeable. It’s not important enough or effective enough to use on their adwords copy. If taglines were good marketing they would be more prominent in current advertising.

    I have tested a lot of marketing ideas over the years, and there are constantly surprises and unexpected winners. I would however never claim to know the best approach – The best approach changes year to year. The experience you get from testing gives you a pretty good place to start from for future tests.

    When your business income is at stake Always Be Testing. Never trust the status quo, and don’t trust your copywriter to give you optimal sales copy (why buy one sales page when you can buy two for twice the price). Test every assumption. ABT.

  • Migrating to Amazon RDS

    I’ve been suffering from some scaling issues lately and since my system administration skills are fairly rudementary I decided to opt for Amazons RDS services for providing a MySQL server that is properly built with the appropriate block devices and automatically kept up to date by Amazon.

    Moving my existing database over to an Amazon RDS instance was relative straight forward.

    After launching an RDS instance from the Amazon AWS console.  I added my existing security groups to the security groups so they could connect to it.

    Once everything had a launched I was able to easily move my existing MySQL database over to the new server.

    mysqldump dbname -u root -p > dbname.sql
    cat dbname.sql | mysql --host=hostname -u dbusername --password newDBname
    

    Once that had run I was able to update my django application to connect to the new server and all was good.

    Surprisingly painless. Thanks Amazon.

  • An Easy VPN on EC2

    I’ve become a little bit more paranoid over the last few weeks about computer security. I used to think I will be fine – why would anyone want to hack my computer and find my personal files. But after a bit of reading I’m more concerned about the far more likely situation of mass spying and infiltration programs either churning through my network traffic or infecting my computers with a botnet. In those cases people will collect everything they can and data mine it afterwards.

    Starting a VPN should help greatly with preserving my privacy.  It also gives me the added benefit of having an IP address in the USA which allows me to watch Hulu and other US only content sites.

    After several false starts trying to get OpenVPN working I decided to try something easier.

    Starting with an existing AMI was easy.  I searched for and launched a new instance using ami-744c971d which is pre-configured for pptp with a simple username/password.  Create a new security group for the instance when configuring it.

    Once the server is launched and running you’ll need to open up some ports so that it is accessible.  PPTP uses TCP port 1723.

    In the Security Group settings open up TCP port 1723 and TCP port 22 (SSH).

    Ssh to the server using the key pair you selected when starting the instance and the username ‘ubuntu’ to edit the settings file.  Edit the file /etc/ppp/chap-secrets  change the username ‘seamus’ and password ‘wiles’ to something else.

    Restart pptp: ‘sudo service pptpd restart’

    You should be able to connect then to the new VPN server.