Archives For dotastro

botastro

Hubot is an open source chatbot created by GitHub. It’s used by various companies, groups, and other techie types, to control systems, gather information, and put moustaches on things – all via chat interfaces. Hubot can be adapted to work via IM, GTalk, Twitter, IRC, and other platforms. ‘Chat Ops‘ is a growing trends, and because it is simple, and quite charming, I think it may stick around.

I’ve just finished an epic few days at the sixth .Astronomy event. This is my own conference series, and I’m gleefully exhausted from several days of talking, making, and hosting my favourite event of the year. More on that in a later post. During the .Astronomy 6 Hack Day (sponsored by GitHub in fact!) I worked on making an astronomical Hubot – which I’ve called ‘botastro‘ in honor of the #dotastro hashtag from .Astronomy itself.

@botastro exists only on Twitter (for now) and to interact you just tweet it. For example if you tweet

then @botastro will reply

You can can send multiple messages to the bot, but I have a growing list of other ideas too. Currently you can say things like:

  • @botastro sunrise Chicago
  • @botastro apod me
  • @botastro galaxify hello world
  • @botastro fun fact
  • @botastro moonphase
  • @botastro tell me about Jupiter
  • @botastro show me Perseus
  • @botastro gif dog
  • @botastro exoplanet me

Asking botastro to ‘galaxify’ something results in text made up of galaxies from Galaxy Zoo (thanks Stuart Lynn!) which is pretty

botastro_2014-Dec-11

and asking it to ‘exoplanet me’ gives you an exoplanet from the catalogue (thanks to Dan Foreman-Mackey and Geert Berensten). The results you get when asking it to show you something or tell you about something are sourced from Stuart Lowe‘s lookUP service, and the space gifs come from Giphy.

These may be silly and fun, but more complex actions become possible – especially once I get a bit more used to Coffeescipt, the language this bot is written in.

@botastro is open source (on GitHub, naturally) and I’d love it if people wanted to add functionality. If you want to try, you’ll need to fork the repo, create a new script, and submit a pull request. Hubot is outlined here, and you can look at botastro’s other scripts for examples too.

Since 2008 I have been running .Astronomy, which is a meeting/hackathon/unconference that aims to be better than normal meetings and to foster new ideas and collaborations. It’s a playground for astro geeks that is more specific than a general hack day, but way more freeform that a normal astronomy meeting. At .Astronomy we have developed into an amazing community.

I know people that have gotten jobs because of .Astronomy, changed careers because of .Astronomy – or even left astronomy because of .Astronomy (in a good way!). We have evolved into an interesting group, with a culture and way of thinking that we take back to our ‘real’ jobs after each event.

In short: it works. Now I’d like to work out how to spread the idea into more academic fields. We’re looking for people in other research areas, such as economics, maths, chemistry, medicine and more.

Adler Planetarium

I have funding from the Alfred P. Sloan Foundation to bring a handful of non-astronomers to this year’s .Astronomy, in Chicago at the amazing Adler Planetarium (December 8-10). The aim is to meet up at the end, and discuss whether you think it could work in your own field, and what you’d need to make that happen. If you’re a researcher, who isn’t an astronomer, and you think this sounds great then that could be you! We have funding to pay for flights, hotels and expenses. It will be a lot of fun – and despite the astronomy focus of the event, I think most researchers, with a bit of tech experience, would get a lot out of it.

If you’re interested then fill out the short form at http://bit.ly/dotastromulti or email me on rob@dotastronomy.com for more information. We are following a formal selection process, but we’re doing it very quickly and will decide by Nov 7th, to allow enough time ahead of the event to make travel plans and such. So don’t delay – do it now!

If you don’t think you’re the right person for this, then maybe you know who could be. If so, let them know and send them to http://dotastronomy.com/about/astronomy-6-multidisciplinary-program/ for more information.

The latest issue of Astronomy & Geophysics includes an article by your truly about the GitHub/.Astronomy Hack Day at the UK’s National Astronomy Meeting in Portsmouth earlier this year.

The projects resulting from hack days are often prototypes, or proof-of-concept ideas that are meant to grow and expand later. Often they are simply written up and shared online for someone else to take on if they wish. This ethos of sharing and openness was evident at the NAM hack day, when people would periodically stand up and shout to the room, asking for anyone with skills in a particular area, or access to specific hardware.

Take a look here: http://astrogeo.oxfordjournals.org/content/55/4/4.15.full?keytype=ref&ijkey=kkvGWSg3ABbIy5S

Martian Nyan Cat

Martian Nyan Cat

publications

Executable papers are a cool idea in research [1]. You take a study, write it up as a paper and bundle together all your code, scripts and analysis in such a way that other people can take the ‘paper’ and run in themselves. This has three main attractive features, as I see it:

  1. It provides transparency for other researchers and allows everyone to run through your working to follow along step-by-step.
  2. It allows your peers to give you detailed feedback and ideas for improvements – or do the improvements themselves
  3. It allows others to take your work and try it out on their own data

The main problem is that these don’t really exist ‘in the wild’, and where they do they’re in bespoke formats even if they’re open source. iPython Notebook is a great way of doing something very much like an executable paper, for example. Another way would be to bundle up a virtual machine and share a disk image. Executable papers would allow for rapid-turnaround science to happen. For example, let’s imagine that you create a study and use some current data to form a theory or model. You do an analysis and create an executable paper. You store that paper in a library and the library periodically reruns the study when new data become available [2]. The library might be a university library server, or maybe it’s something like the arXiv, ePrints, or GitHub.

This is roughly what happens in some very competitive fields of science already – only with humans. Researchers write papers using simulated data and the instant they can access the anticipated data the import, run and publish. With observations of the Cosmic Microwave Background (CMB) it is the case that several competing researchers are waiting to work on the data – and new data come sour very rarely. In fact that day after the Planck CMB data was released last year, there was a flurry of papers submitted to the arXiv. Those who got in early, likely had pre-written much of the work and simply ran their code as soon as they had downloaded and parsed new, published data.

If executable papers could be left alone to scan the literature for new, useful data then they could also look for new results from each other. A set of executable papers could work together, without planning, to create new hypotheses and new understanding of the world. Whilst one paper crunches new environmental data, processing it into a catalogue, another could use the new catalogue to update climate change models and even automatically publish significant changes or new potential impacts for the economy.

I should be possible to make predictions in executable papers and have them automatically check for certain observational data and automatically republish updated results. So one can imagine a topical astronomy example where the BICEP2 results would be automatically checked against any released Planck data and then create new publications when statistical tests are met. Someone should do this if they haven’t already. In this way, papers can continue to further, or verify, our understanding long after publication.

SKA Rendering (Wikimedia Commons)

SKA Rendering (Wikimedia Commons)

This is high-frequency science [3], akin to high-frequency trading, and it seems like an interesting approach to some upcoming data-flow issues in science. The Large Hadron Collider (LHC), Large Synoptic Survey Telescope) LSST, and Square Kilometre Array (SKA) are all huge scientific instruments set to explore new parts o the universe and gathering huge volumes of data to be analysed.

Even the deployment of Zooniverse-scale citizen science cannot get around the fact that instruments like the SKA will create volumes of data that we don’t know what to do with, at a pace we’ve never seen before. I wonder if executable papers, set to scour the SKA servers for new data, could alleviate part of the issue by automatically searching for theorised trends. The papers would be sourced by the whole community, and peer-reviewed as is done today, effectively crowdsourcing the hypotheses through publications. This cloud of interconnected, virtual researchers, would continuously generate analyses that could be verified by some second peer-review process; since one would expect a great deal of nonsense in such a setup.

When this came up at a meeting the other day, Kevin Page (OeRC) remarked that we might just be describing sensors. In a way he’s right – but these are software sensors, built on the platform and infrastructure of the scientific community. They’re more like advanced tools; a set of ghost researchers, left to think about an idea in perpetuity, in service of the community that created them.

I’ve no idea if I’m describing anything real here – of it’s just an expression of way of partially automating the process of science. The idea stuck with me and I found myself writing about it to flesh it out – thus here is a blog post – and wondering how to code something like it. Maybe you have a notion too. If so, get in touch!

———-

[1] But not a new one really. It did come up again at a recent Social Machines meeting though, hence this post.
[2] David De Roure outlined this idea quite casually in a meeting the other day, I’ve no ice air it’s his or just something he’s heard a lot and thought was quite cool.
[3] This phrasing isn’t mine, but as soon as I heard it, I loved it. The whole room got chatting about this very quickly so provenance was lost I’m afraid.

.Astronomy 5: What’s Next?

September 20, 2013 — 3 Comments

The .Astronomy 5 Unphoto – Credit: Demitri Muna

As the fifth .Astronomy came to a close on Wednesday, I felt as I always do at the end of these meetings: tired, emotional and super-excited. It’s hard to explain the energy at these events. There is something almost magical in the air as the participants ‘click’ (usually about an hour in) and then begin talking, making and doing great work.

.Astronomy is about actually doing something. As Kelle Cruz and I remarked yesterday – we like ‘people that do shit’. At .Astronomy you feel that if someone has an idea we should just all try and make it happen. It could be the best thing ever, and failure is just a chance to learn. It’s not a common attitude in astronomy and it’s certainly difficult for many early-career people to think that way.

I’ve always been lucky. My PhD supervisor was very willing to let me try crazy things (he let me get distracted by creating .Astronomy for a start!). At the Zooniverse we have spent years now, just pushing code live and making new things. They’re not always perfect, but we learn every time and we have left a trail of marvellous creations on the way. Each new thing learns from the last.

We also absorb the ideas of others quickly, and encourage collaboration with new people. It’s this approach that led to the creations of some of our most interesting projects recently, such as Snapshot Serengeti, the Andromeda Project and Space Warps.

During his Keynote talk Tony Hey (Microsoft Research) showed a quote I’ve not seen before.

“If you don’t like change, you’re going to like irrelevance even less.” – General Eric Shinseki, retired Chief of Staff, U. S. Army

I think I might put this on my wall. It sums up perfectly how I see much of science and could easily be the motto of .Astronomy. Tony’s keynote was brilliant BTW and you can see it here. Tony spoke about the Fourth Paradigm and told the tale of how the availability of astronomical data led to the SDSS SkyServer, which sparked the creation of Galaxy Zoo, which sparked the Zooniverse. In a way, .Astronomy was partly sparked by Galaxy Zoo too.

The folks at .Astronomy have built many projects that embrace the web fully, with an ethos of sharing and participation. These projects are changing the way astronomy and outreach are done: Chromoscope, 365 Days of Astronomy, AstroBetter, Astropy, astro,js, and the Seamless Astronomy groups ‘Bones of the Milky Way‘ paper; there are more but these are excellent examples.

So after .Astronomy 5 I’m left wondering where to take it next in order to facilitate more of these projects. There were 40 hack day pitches at this year’s event. There were so many hack day reports the follow day (the 2-3 minute slots where people show off their results) that we had to over run into coffee and use up most of lunch time too. Many of those hacks will, I hope, soon be appearing on the .Astronomy blog when people have time to write them up. Some of them are already popping up on GitHub (e.g. d3po).

The other wonderful thing about the meeting was how it once again encouraged genuine debates and discussions that sound like they might actually lead to change. The unconference sessions on diversity in astronomy went beyond the usual format and did not fall in to the trap of collectively preaching to the choir. A document has been drafted with actionable ideas. I hope it is revisited soon. Similarly sessions of the future of academic publishing were not bogged-down in the usual complaints but actually became a real debate about practical things we could do differently.

There were also highly informative unconference sessions that would not have happened elsewhere; enthusiastic tutorials of Astropy, Authorea and the merits of combining noisy classifiers are all jumping to mind. These meetings organically emerge from the crowd at .Astronomy and they’re, interactive, productive, and brilliant.

So as I ponder on the future of .Astronomy (I’d love your thoughts) I’ll leave you with some of the wonderful video hacks that were produced at this year’s event. Don’t Call Me Colin is a song about a sad exoplanet from Niall Deacon, Emily Rice, Ruth Angus and others. There is also a timelapse of .Astronomy itself in action from Amanda Bauer.

Thank you to everybody who took part, gave their time to talk, help organise the event; and who followed along online. It was a great meeting and I’m already looking forward to the next one. Long live #dotastro!

During the Perseid meteor shower, I blogged a video of a bright meteor taken by astrophotographer Mel Gigg. He had shared the image fairly widely and soon others noticed that they had caught the exact same shooting star themselves. In fact four observers had caught the same object as it flew into the atmosphere above Southern England, three of them have shared their images online (Wayne Young, Mel Gigg and Steve Night).

Credit: Wayne Young

Credit: Wayne Young

Credit: Mel Gigg

Credit: Mel Gigg

Credit: Steve Knight

Credit: Steve Knight

Look carefully and you’ll see that these images show the same streak of light but against drastically different star fields. That’s because meteors are high above the ground and visible across a large area. Due to the effect of parallax, they appear to shift relative to the night sky for different observers. In the extremes, an observer underneath the meteor, would see it go directly overhead, whereas others might see it from the side, where it would appear to fly nearer to the horizon. In this case it was seen by four people from different positions so they each had a different angle on the meteor and a different backdrop of stars.

In a wonderful example of citizen science, Wayne Young (one of the four photographers) took the four images and the lat/long data of each observer’s location, and created a 3D model of this particular Perseid’s path. You can see it below modelled in Google Earth (KML file here).

To create this he’s triangulated the path of the meteor by comparing each of the four images to one another. Given the capabilities of computer vision tools and astrometry.net, I wonder how much of this could be automated. It wouldn’t be hard to search Flickr for shooting stars seen at similar times and locations maybe we can scrape more trajectories automagically? This might be an ideal hack day project for .Astronomy. To plot many of these paths on top of each other would be interesting.

It’s fun to surmise that, given this Perseid’s path, it would have touched down in a field in North Devon. Good job it most likely disintegrated long before then.

We’re running the fifth .Astronomy conference later this year in Boston. .Astronomy is a small (and awesome) conference for astronomers, where you must apply to participate. Although the tone is relaxed, spaces at the event are in short supply (there are only 50 places). You don’t have to talk at .Astronomy, and there are only a few speaking slots, but it’s a pretty friendly crowd and you can talk about a wide variety of things. So why did only 2 women submit an abstract (out of 27 female applicants) versus 30 men (out of 65)?

.Astronomy 5 Signup Gender Ratio

We would like to create a broad group of speakers but it’s hard to select talks that don’t exist. Did we inadvertently create a bias toward male speakers by soliciting abstracts on the sign-up form? If so, that’s a worry because it’s how a lot of conferences do this.

To be clear: on our simple conference registration form, almost 50% of men submitted an abstract, but only 7% of women. Holy moly.

There has been a great deal written about the fact that women lack self-esteem, relative to men. This explains some of the gender gap in pay, promotions and even published op-eds. This isn’t news, really. In fact the 7% and 50% figures above are eerily close to the percentages for each gender who negotiate starting pay after getting an MBA – and that study is more than 10 years old!

.Astronomy Signup Gender Gap

What is news to me is that we committed the same error at a progressive conference in 2013. Does this mean that conference registration forms like the one used for .Astronomy are an example of unwitting bias against women in astronomy – and who knows: science, academia, conferences in general?

I’d be interested to know how this plays out at other conferences and events. Do the UK National Astronomy Meetings see a similar gender gap? Do AAS Meetings? Does anyone else have anecdotal examples of similar or contradictory things happening?

I realise that there are women with plenty of self-confidence – and also men who lack it. I also realise that self-confidence does not correlate with academic ability and so perhaps we need a better system for selecting people for talks, promotions or jobs. I’m not proposing any solutions here – that would be extremely self-confident of me. What I do know is that whatever system would improve the situation, it will also be important for the women of academia to boldly go where statistically fewer women have gone before: and submit more abstracts.

As for .Astronomy: if you’re coming to the meeting in September and you’re a woman who didn’t submit an abstract (there are many of you!) then feel free to email one to me now. The SOC are still picking a range of speakers and talks, so we’d love to hear from you.

[There’s a follow-up post to this here]

Unproceedings of .Astronomy 4

.Astronomy Hack Day NYC

December 12, 2012 — Leave a comment

On Saturday, New York astronomy geeks will convene at the offices of bit.ly for the first .Astronomy Hack Day! We do hack days as part of the main .Astronomy events and people have often asked us to do hack days outside of the main conference series. So when August Muench began saying he’d like to run such a thing in NYC it was very exciting. 

You can still sign up on Eventbrite (although you may be added to a waiting list at this point – the plan is to widen the event and then let the waiting list in.) If you’re part of the geek elite of astronomy then I highly recommend going along. It’s just for the day and just for fun.

Go and make something, create. Share ideas, and meet the astro geeks in your area. I know so many projects and working relationships that have come from Hack Days of all kinds – they can be really productive events.

I wish I could be there, but I can’t. So I’ll just have to rely on you all to go and make this .Astronomy Hack Day awesome. I’m really looking forward to seeing what comes out of it.

.Astronomy 4

June 24, 2012 — Leave a comment

In 2008, in the midst of my PhD, I ran a conference called .Astronomy. The idea was to bring together all the other astronomers who were into the web and networks. It might not seem very long ago, but 2008 was before Twitter went mainstream and before everyone’s gran was on Facebook – to use one measure. We had data scientists, robotic telescope people, bloggers and others present (both in person and remotely).

The conference, which I held in my home institution in Cardiff, was a lot of fun and I got to meet a bunch if cool people. I was surprised when a clutch of the attendees came to me afterwards and asked if they could help run the next one.

Together we have gone on to organise three further, awesome events. Following Cardiff (Sep 2008) we had .Astronomy 2 in Leiden (Dec 2009) and 3 in Oxford (Apr 2011). In two weeks I’ll be attending .Astronomy 4 in Heidelberg. It’s being held in the newly built Haus der Astronomie, which is shaped like a spiral galaxy and looks gorgeous!

Haus der Astronomie
Haus der Astronomie

The event – I hate calling it a conference, as it feels like more than that – keeps improving, and evolving from meeting to meeting. They are all very informal affairs, and quite intense. I’m getting very excited about #dotastro 4, as it is known on Twitter. I’m really grateful to Sarah Kendrew for making the Heidelberg event happen this year. Being the local go-to person is a massive job, and she’s doing it fabulously!

One thing we’ve added this time is a shared, public Wiki for the participants. In just a few days it has already filled up with the crazy and brilliant ideas I’ve come to love and expect from .Astronomy. There are suggestions for a huge range of projects, from an automatic astronomy poetry generator to a service that can use the entire astronomical literature to generate new, unexpected hypotheses. The ideas on the wiki are all about trying to get the most out of the Hack Day, when everyone is free to build/create whatever they like with the others at the event.

.Astronomy is all about the relationships between the people who attend. We keep the event small, try to keep people in close proximity, and aim for an informal few days. I’ve gotten to know some of my favourite astronomers through .Astronomy and I know several working relationships that have emerged because of it. I have high hopes for number 4, and the buzz around it is growing. I think it’s going to be awesome.

You can learn more at http://dotastronomy.com follow us on Twitter @dotastronomy and follow all our fantastic Heidelberg participants via our conference4 list.