Mutt: Delete/Move thread to "Deleted Items"

folder-hook . 'macro index d "<save-message>=Deleted<quote-char> Items<enter>"'
folder-hook . 'macro index \cd "<untag-pattern>.<enter><tag-thread><untag-pattern>~D<enter><tag-prefix><save-message>=Deleted<quote-char> Items<enter>"'

PyCon UK 2014

Hello, I'm never going to remember everything until next Monday, so have a braindump. I'll put up an item for questions in that meeting. These are the talks I went to, I've linked slides for some but not all. Note PyPy is an alternative Python interpreter; PyPi is where pips come from; PiPy is a Raspberry Pi lib; and if you name your next lib PiPi I will hurt you. --- Keynote: Ecosystem threats to Python (Van Lindberg) Java, JS and Go. These are threats because Python has poor interop with them. He did mention it's not a zero-sum game. There wasn't a rallying cry or solution offered, Van just called these out. There was a lot of instinctive dislike of Java. It got called out as a 90s ancient and enterprise-y. There didn't seem to be much understanding of why people might choose the JVM as a platform. Scala got a brief mention at least. Java is the native language for Android, I can't remember if Van mentioned this but Python is shut out of mobile right now. JS was lumped with node.js. CoffeeScript, PureScript etc weren't mentioned. I don't remember much beyond a few cracks really. There was some admiration for Go, particularly the single-executable deployment story. Speaking to Michael Foord (Go dev at Canonical) afterwards, Go bucks the trend on exception handling and several libs try to fix that. The performance is good though - Van has a fork of Swift with pieces rewritten in Go and he says it's 3x faster. --- HTTP/2: because the Web was too easy (Cory Benfield) https://t.co/Eq7WZ96wDI - slides Semi-recent developments in web dev have included CSS spriting, optimising load time for 'above the fold' and aggressive minification + concatenation of resources. Many of these are essentially hacks to mitigate the misuse of TCP connections by HTTP. TCP connections are intended to stay open for minutes or at least seconds, not milliseconds. Not only do these hacks complicate the toolchain, they make caching more difficult. HTTP/2 offers great latency reduction by using a single connection for multiple resources. It also allows servers to push resources you didn't yet ask for (EG CSS, JS) so they're in your cache once you've parsed the HTML. The requests lib already supports HTTP/2, so from an OpenStack perspective, when we want to switch it should be fairly easy. However it is ~95% SPDY which wraps HTTP/1.1 and was very much designed for browsers. Benefits to RESTful interactions are definite but incidental to the design. There is some talk of an HTTP/3 which tackles this but it sounds like wishful thinking at this point. --- Ganga: an interface to the LHC computing grid (Matt Williams) Map/reduce for the LHC. You submit jobs, either to a local backend or the LHC grid, and it does them for you. They have a remote filesystem so you don't have to download gigabyte datafiles locally. Pre-dates OpenStack. --- Advanced py.test fixtures (Floris Bruynooghe) http://pytest.org/latest/contents.html Py.test looks interesting. I can't say for certain it's better than our current subunit/testr/mock/tox stack or even how many of those it replaces. It's hard to imagine it's any worse though. The fixtures offer: * autowired dependency injection * parameterisation - multiple tests are generated so you know which values failed * finalization * markers - general tagging mechanism, you can restrict a test run to EG not run DB tests. You can reflect on markers from fixtures, to EG skip tests if a server is unavailable (and fail if the test runner is CI). Unrelated to PyCon, Matt W found http://pythonhosted.org/behave/ for BDD. Matt + I have both done BDD previously and found it beneficial. --- Keynote: Lessons from Strangers (Rachel Sanders) http://www.slideee.com/slide/pycon-uk-2014-keynote * "People are happiest when they can get their work done." This is an MBA-level truth bomb. * People don't want to use your product, they want to HAVE used it. * Nobody wants to read the damn manual. Or can remember 850 pages. Build clean APIs. Do UX. Listen. Empower. Do RCA. Plan for mistakes. * Read "The Design of Everyday Things", by Don Norman (on my Amazon wishlist now!) --- When Performance matters (Marc Andre-Lemburg) --- The High Performance Python Landscape (Ian Ozsvald) https://github.com/egenix/when-performance-matters Ian gave me (not just me) a free book! If anyone wants a browse, it's on my shelf. Step #1, as any fule kno, is to profile. Some tools: cProfile, line-profiler, memory-profiler. Run Snake Run is a visualizer for cProfile. There are some simple optimisations that speed up regular Python code. For example list comprehensions are faster than for loops; #join is faster than string interpolation (duh); string dict lookups are slow, use ints or interned strings; exceptions are terrible if triggered (~625ms not ~40ms). Note this is all for high performance Python, not necessarily the Pythonic or readable way to do it. The next step is using C extensions for your tight loops, EG looping in numpy/lxml; the operator module; cStringIO. You can save memory using slots or the Flyweight pattern (Design Patterns: not just for Java!). Beyond this... PyPy is a Python interprester with a JIT, which makes it much faster than CPython (the regular interpreter). It supports cffi (so you can keep using C extensions like numpy) although the callout to these is slower than from CPython. Note PyPy is *slower* than CPython until the JIT kicks in. Cython can compile your Python to C but you need type annotations which make it no longer Python. You can autogenerate annotation comments with ShedSkin, which could be manually converted to Cython types. Or you could use Pythran which uses the comments directly. However, the numba lib manages equivalent (iirc) speedups using decorators to mark functions for JIT. I don't see why anyone would use Cython or Pythran, JITs rule OK. If your problem is vectorizable, you have a nuclear option in OpenCL. You can use that on your CPU or GPU. --- Simulating Quantum Systems in Python (Katie Barr) http://physik.uni-paderborn.de/?id=178571 (not from KB) I met Katie while getting some air. She was quite irate about HP cancelling their quantum research programme! Her talk was on simulation of discrete-time quantum walks across a 2D array. Although the science part was a bit boggling, the Python part was pretty simple. The probability wave spreads across the array, attenuating as it extends but building interference patterns. One location, marked by the coin flip used there being unfair, builds up a high probability. The discrete-time quantum walk can find this location in O(log(n)) iterations, which is provably minimal. --- Use of OpenStack CI for your own projects (Yolanda Robla Mota) Yolanda from the Gozer team gave a comprehensive if whirlwind tour of Gerrit, Zuul, Jenkins, JJB et al. Not many people turned up to be honest, I think it went way over most people's heads and/or needs. --- The IPython Notebook is for everyone (Gautier Hayoun) http://opentechschool.github.io/python-data-intro/core/notebook.html https://cloud.sagemath.com/ - try it out (not GH's links) The IPython Notebook is a web UI to the IPython shell. This means you don't want to make a server publicly accessible! However it's pretty fantastic for playing with "what if?" scenarios or creating tutorials. I can see great application in classrooms, where you put a prewritten notebook on each student's PC and let them tinker with it. --- Stormy Webber (Wes Mason) I missed most of this talk but essentially Wes presented Tornado, a web framework and async networking lib. Think node.js for Python. --- Functional Programming and Python (Pete Graham) Unfortunately Pete talked mostly about the advantages of functional programming, with little proof or demonstration. Functional programming did not feature heavily in conference discussion at all. --- Building great APIs in Python (Paul Hallett) a) Use REST; b) keep it simple. I did learn about the PATCH verb, which is what you should use to PUT a partial resource IE do a partial update. HATEOAS - horrible acronym for using URIs to identify resources in REST APIs instead of UUID, numeric ID or common name. Tastypie is a neat lib for making resources available, offering an automatic endpoint list. Httppy gives you a shell from which you can GET resources easily. --- How does a spreadsheet work? (Harry Percival) Harry talked us through Dirigible, which has been open-sourced. I think he said it was going to be a product but PythonAnywhere decided there wasn't a market. He agreed that in practice, you should probably just use IPython Notebook. It's generally pretty simple, he started out just eval'ing formulae then making cell value substitutions. You build a tree to handle dependencies. Engaging, fast-paced speaker, his site is obeythetestinggoat.com and he wrote O'Reilly "TDD with Python". Worth seeing just for entertainment. --- Keynote: A time traveler's guide to Python (Jessica McKellar) (I've tweeted her to ask for slides, not available yet afaict) I thought this was going to be a fluff talk initially. Jessica spent 20mins talking about Python's history then she blew us all away. Turns out she's a startup founder, a PSF director (obviously an engineer) and would like the Python community to shut the hell up about 2 vs 3. Instead she would like us to focus on growing market share through the traditional mechanisms of a startup. For example, a focus on the onboarding experience and seizing opportunities to push Python EG in schools. I was slightly creeped out by zero-sum language but I trust it was just phrasing. It was refreshing to hear someone applying time-proven business lessons on growing adoption to a programming community. Jessica mentioned mobile as something to talk about. Initially she didn't offer an opinion but after being asked directly she said she'd like to see something. --- Using Python to improve government (Michael Brunton-Spall) https://speakerdeck.com/bruntonspall/using-python-to-improve-government-pycon-uk-2014 I first (and last) saw MBS at FPDays, talking about Scala at the Guardian, so I was interested in his shift. He spent a lot of time debugging the JVM GC apparently and had strong moral reasons for wanting to work in the public sector. His talk was inspiring, he was hired by the Cabinet Office to help sort out govt IT. The DirectGov initiative has had some success in publishing but MBS was asked to help with transactions. He first worked with the Insolvency Service. He took a .NET team and, with the help of a big Cabinet Office stick, took them right out of their comfort zone to Python. This stopped them retaining any bad practices without being too hard (Haskell). He taught Agile, storyboarding, composition, decomposition, recursion, UX and many other things. Specifically they worked on the redundancy payments system. He showed an interaction diagram from hell, it seems the service user had to submit an initial 16-page form then have several follow-on interactions. Bearing in mind similar points to Rachel Sanders' talk, and that people claiming redundancy benefits really do just want to get paid not fill out forms, they converted this into a wizard with recurring sections, usually only asking for 2-3 pieces of info per page. Finally he brought the whole team to PyCon! They all seemed engaged and really proud of the process and results. --- The Minecraft Challenge (Katie Bell) Katie's challenge was to create a way to script Minecraft that preserved the nature of the game, that was approachable and allowed multiple participants at once. She achieved this by creating a scripting interface to control a robot. She had a few demo scripts, EG to mine for iron or build a stone hut. She also dropped her robot pumpkin on a chicken. Feathers and applause everywhere (the robot is represented in game by a pumpkin). An important part of her message to children is that when you get bored of something, you can script it. The robot style of scripting supports this goal of playing the game, just more efficiently. The (more powerful) random access scripting style on the other hand makes it a different game. --- Dr. Jython; or, How I Learned to Stop Worrying and Love the JVM (Naomi Ceder) http://j.mp/1rlhaqu Contrary to the title, Naomi did not seem to love the JVM. The talk was a bit agonising for a Java native; she complained about Ant (well yeah, Ant is a bad time in a box) but preferred Make and managing her classpath with shell scripts. She complained about how hard it was to pip install packages instead of simply embracing JVM libraries. Finally she compared bad Java code against good(ish) Python code, which is unfair and annoys me. On the plus side, she achieved what she wanted to, which was talking to a SOAP service with buggy WSDL. This despite the latest released version of Jython being 2.2 (she used the beta 2.7). She did mention there's no GIL, hence no need for b****y eventlet. --- Keynote: Miss Adventures in Raspberry Pi (Carrie Anne Philbin) Carrie talked about taking Raspberry Pis into schools to teach Python with. I spent a lot of the talk hoping she'd mention Barefoot, she did mention CAS. She talked about Scratch and how although it was great it was also important to graduate students to text languages. --- Miscellanea https://www.nanoporetech.com/ had a stand with several DNA sequencers, each ~$1000 and smaller than a phone. Seemed worth mentioning. http://bazaar.launchpad.net/~ubuntuone-hackers/conn-check/trunk/view/head:/README.rst conn-check knows how to check that all sorts of services are available. Pyfakefs is good for stubbing file IO. http://python-namibia.org/ is a thing. Unrelated to PyCon but completely badass: http://t.co/MysUpW7mU3 There was basically no discussion of static type systems whatsoever. I was pleased to meet not one but two prototypical PyLadies. We'll see if a group can form. Unfortunately one just started a job and the other can't code yet.

Angeland

Once upon a time, when we flew on wings of steel and nested like termites; when every man, woman and child could consult the wisdom of ages. An ancient race came to Urf and built a castle on an island. The walls of this castle reached further than the eye could see, and it would take a man half a year to circumnavigate. Assuming he could walk on water, for the moat of the castle was the sea itself. The castle was called Angeland.

The people of the day, sick with fear and pride, cursed the ancients. They used their wings of steel to hurl fire and hate against the castle, but their wings were torn from them. Swarms of men crashed against the castle, up the walls and into the sea, but their hate fell from the sky and boiled their flesh.

Then one day the door opened. The sun beat cruelly and the land scorned their pleas, so the men and women were hungry and entered Angeland.

= Floor 0. Eden.

Lidar and Oogle were free for the day. Their teacher, Feo Rosefelt, had entered the Maze yesterday and their school hadn't managed to arrange a substitute.

(They explore and discuss the Maze)
(They wander into a forbidden area and the walls begin to move. They hear faint cries and sacrifice their chance to leave moving towards them. They find a suspicious girl called Anghun.)
(They begin to explore the Maze. They are hunted by giant boar in a forest, nearly freeze crossing a mountain pass in a blizzard and are betrayed resting in a city of charcoal. Throughout this, hellish creatures pursue them and occasionally tear each other to pieces, although the children successfully evade them)

(The number 235235 becomes a theme, through puzzle solutions and chance arrangements of features. Usually it requires conferral to discover the full number. Many other six-digit numbers also crop up, all under 240000)
(Eventually they reach the end. 24-hour news plays on vidscreens on the walls; two years have passed and their home has been obliterated by war. A large crystal glows in the center of a vast room, condensation streaming down the sides. Periodically individuals or small groups go up to the crystal, place their palms on top and disappear.)

(Lidar and Oogle want to use the gate at 23:52:35, but Anghun refuses. Reluctantly, they enter on time and reach the next floor.)
(They are immediately faced with an angel. The angel offers them a devil's choice: serve the ancients together, or rule the Maze but forget each other. They choose to serve and are graduated to Maze Architects.)

= Floor N. Prime.

They meet Buds, who communicates through text message rather than remove his headphones. Buds introduced the timestamp system, intended to encourage only those who could agree to enter the gate together. An unfortunate side-effect is of encouraging those who enter solo to take the lower choice. They begin to watch maze runners and take particular interest in a pair not unlike themselves. Unfortunately, Anghun, transformed into a demon, has retained enough intelligence to find ways around the rule against demons directly harming runners. She taunts the boy into hunting her, then scares the girl into searching for the boy. While they are divided, she steals their food or equipment and they are eaten.

???
Lidar and Oogle become magical ponies and spread rainbows everywhere. The ancient ones finally die out, satisfied the universe is in good hooves. The End.

Gridmole reborn

I downloaded Unity, it's pretty fun. Managed to crash it once already! Turns out update scripts get called a lot; creating one object per call makes a lot of stuff.

The plan is to rewrite Gridmole under the new name Gridslam, with a 3D engine, particle effects etc. Oh and respond to more than one mouse button. Doing everything from scratch in pure JS/Canvas was extremely slow so I'm going to try Unity3D.

The original concept was a healing simulator. However I'm not that interested in a fixed range of 'spells'. I'm thinking I'll use a dragbox / holdclick model with an exponential efficiency curve and an S-curve for power (with a bump at the start).

Given Power (in points out of 100), Time (in seconds), Cost (points where click = 50), Efficiency.

Click: power 50, time <0.3, cost 50, E = 1
  - "Flash Heal"
Held click: X = (min(T,3)*10/3)**2, S = 0.25. Power X, cost = P - X*S*P/100
  - Power and efficiency both increase as you hold up to 3sec
Dragbox: S = (25+N)/100. P/N = X,
      C = N(P - XSP/100)
        = N(XN - XSXN/100) 
        = XNN(1 - XS/100) 
        = XNN(1 - X(25+N)/10000)
  - Power is equal for all targets, efficiency increases per target.

As a future improvement, I could add shield mode (-10% effi, adds secondary HP pool) and HOT mode (healing is delivered over T**3 secs, for some efficiency bonus).

So far this is a very smooth design. It needs some bumps to create pressure. The bumps will come from irregular incoming damage. Levels will consist of a number of Hurts, with an attack pattern (EG row, column, random(5)), frequency (probability of occurrence) and power level. With this degree of randomness, I'll need to cap the short-term DPS or (very) occasionally the system will go totally psycho.

When blocks take damage they will wobble and overextend towards red before returning. I'd like to add warning indicators to Hurts, which would make the shield mode more interesting and allow for prehealing. Ideally this would be meteors and dragons, but I'll settle for a red backdrop for now.

TBC

Roller life

With reference to: http://spectrum.ieee.org/automaton/robotics/robotics-software/042910-a-robot-that-balances-on-a-ball

It's hard to imagine a wheeled creature evolving. Scifi provides few examples, I believe David Brin touched on it (along with some very intriguing waxy-toroid creatures).

Let's imagine a Rolla bird. It is flightless but egg-laying, with thick fluffy feathers. The baby Rolla hatches in late Spring, as the first rays of sun strike its icy habitat and begin the thaw. It immediately lays an egg (unfertilised), half the size of the bird itself, and makes its first attempts at balance. As the thin snows clear, revealing the smooth glacier ice formed over polar volcano flows, movement becomes easier.

The Rolla sets out with its siblings in search of food. A pygymy spruce local to the area offers up pine nuts, just within reach of the infants. The pine nuts are difficult to digest for the small stomach of the baby bird and a substantial part is injected into the egg under its feet.

As the Rolla matures, this contribution is enhanced with additional enzymes and organisms. A nucleus forms within the mass of waste which becomes like a second stomach. The hardest parts, mostly bits of cone, are pumped outwards through a constantly extending network of cracks to expand and reinforce the egg. Softer parts which still carry nutrition are retained in the centre.

After a couple of months the Rolla has reached adulthood and migrates in search of a mate. Rollas are hermaphroditic and exchange ova while still upright. The foreign ova, injected into the egg, is fertilised by the DNA therein and begins to divide. The by-now thoroughly decayed soft food waste is consumed to fuel the growth.

No more than a month after conception, the adult Rolla bird lies down for the first and final time. Its frantic summer of feeding is over and winter stands in the wings like Death, beyond survival. As the days draw in, the adult dies and the egg is slowly buried in the snows. The thick outer shell and battery of nutrition will sustain it until the land is inhabitable once again, and this most independent of birds will hatch again.

--

I tried to introduce a couple of evolutionary triggers - a low-hanging food source and a killer winter not even a penguin could survive. The glacier ice is an attempt to explain WTF the place is flat enough for a wheel to work. It's still not the most probable of creatures but not entirely beyond imagination I hope!

The parent who dies before the child is born is rather compelling. Every year a new generation would be totally alone. It'd be interesting to extend this to a small planet, with an extremely long year and a civilizing race. Born in the snowy tropics, slowly migrating to the poles as the equator becomes uninhabitable then back again to breed and die. Each generation would discover the artefacts of the last as if a foreign people. The standard tradition vs discovery dichotomy would have to be reexamined when 'tradition' (IE following the example of prior generations) requires discovery. And of course there is a great, unavoidable extinction every generation.

Crazy cat man

Working towards my lifegoal of being eaten by my cats when I die, I've decided I will need a cat. I tried the Blue Cross initially but they don't have a Bristol centre. Foop. I tried a few others, including one VERY unfriendly site that pretty much said "Halo + mansion or GTFO" - and didn't even have any cat listings up! The RSPCA site was ok but they only have a few cats listed which is a shame as they must have a hundred waiting. I flatter myself that in this net-savvy generation there are an increasing number of consumers who like me are happiest not even contacting a provider until we know exactly what we want!

Fortunately I tripped and fell heavily upon Bristol and Wales Cat Rescue. They work out of people's homes which could a) mean they're big-hearted animal lovers doing their best on the cheap, b) a scam. However they have cat macros on the site! This substantially enhances their credibility. Also they have a cute rollerskating chick. A girl that is, not a baby chicken, though that would be MOST amusing.

I was planning on picking up a little princess but experience must be sinking in at last as I've decided an elder statesman will suit my lifestyle much better. Maestro the 14 year old, barrel chested and fit as a fiddle. Well, barrel shaped somewhere as he is 6kg. And possibly only fit as a very well-loved fiddle considering his heart murmur and the dental work he's just had done. I shall be purchasing insurance for the wee beastie.

On which topic - FOR SCIENCE! At least, a spirited inquiry. Animal Friends insurance is dirt cheap which reviews say translates directly to being treated like dirt. Sainsbury's allegedly double your premium after a year even if you don't claim, and do not necessarily inform you of this despite the terms of the direct debate mandate. Some others are also full of shit. More Than appear to possess something of a monopoly position in the market for "insurance wot pays up" so with their good selves I shall entrust the care of my feline companion.

Gosh I'm long-winded. Brevity may be the soul of wit, but verbosity is the full English breakfast. And so to vittels. Cats are naturally carnivorous in contrast to my addled self. I appear to be pursuing an oxygenarian or at least breadhead diet. Anyway while I'm not about to start force-feeding the poor creature spinach, I'd rather it didn't eat a) anything which hopes to be fois gras in its next lifetime, b) total crap. Item A rules out pretty much anything from Tesco. Item B is a slippery slope which begins with eliminating bacon flavour lard balls and ends with me preparing chicken sushi garnished with intestine and liver pate. Damned if my pet is eating better than me.

Suddenly! Bristol! It turns out one of the better kibbles is available from Roxford's which is about 25mins walk from my house. I'll probably drive since I'll be carrying about 10kg of used animal tissue home. Here's a picture.

So it looks like the scragball will cost about £22 in insurance and £18 in food (70g/day @ £21/2.5kg) per month. Ouch. He may have to subsist on Whiskas or some other proletarian gruel for a little while. On the plus side I may save on hot water bottles.

Go ducky gogogo!

Took about an hour. I made those clouds by hand then just as I'm finishing off I discover GIMP has cloud filters! Oh well, next time. I used some HSV noise and blur filters for the road, which was nice because drawing all those little dots would have taken FOREVER! The skate wheels have some DIY noise on them courtesy of an elliptical mask and the Galaxy brush, but noise filters are so much better.

It's time for bed. Tonight I have researched cat insurance, researched cat food, written a blog post about researching cats and drawn a ducky. Who says you go crazy when you live alone?!

All action non stop internet dragon pwnage!

The only two things I really like about MMOs are new quests and fighting bosses. Quests require this whole persistent world, they're expensive. Screw that.

Let's make a game all about killing bosses. None of the fluff around it. You connect to the lobby. You pick your class, maybe tweak it a LITTLE. Not 12 items with gems and enchants, no way. More "blue Rohan" (fast hitter) or "red Rohan" (heavy hitter). Maybe 10 total skills, roughly half for core rotation the rest for utility.

Gear progression naturally works the wrong way. The game gets easier as you get better. Instead, kills on low difficulty modes should unlock higher difficulty modes. Alongside that, a ladder will control who you are grouped with to avoid that "waiting for everyone to get it" feeling. Since the game will have no persistent world, there's no need to shard the playerbase, keeping wait times low.

Let's make it free to play! Give away maybe 9 bosses in 3 arenas for free. Release a new paid DLC boss every... week? Is that really so much to ask? Offer a discount all-new-bosses sub and heavily discounted old boss bundles. Obviously it'll be easiest to find groups for the new bosses and all the buzz will be around the new bosses.

Let's make it fast! A raid boss is a natural 20 minute encounter. That's snacksize! You can do one before dinner without ruining your appetite (or being late to table). Let's put the tactics video right in the goddamn game, played by the devs, with their kill time for all the world to see. You can skip the video if you want to play blind or try to beat the devs if you're hardcore.

Ladder prizes... hmm... say, get to design a boss? :)

--

The impetus to writing this up was my survey submission to Trion after quitting Rift. I figured hey, even if they steal it without credit, I want to play it! But just in case I'll post the exact text I sent them here.

Make a PvE version of Bloodline Champions. With matchmaking by ladder.
Free to play, new bosses cost, old bosses available in packs.
Progression up difficulty settings, not gear.