PDA

View Full Version : TravianTools Project



Donkeh
01 Aug 2013, 01:49 AM
So I'm going to start a big project called TravianTools.

Currently it will be located at http://jasondover.net/TravianTools/

I'm going to be aiming at bringing everything together - between GetterTools & Killroid's Tools.

I'll be logging any information, new features, and idea lobbying in this thread - so if you have any concerns or requests, please post away.

Donkeh.

Lone Ranger
02 Aug 2013, 04:19 AM
So I'm going to start a big project called TravianTools.

Currently it will be located at http://jasondover.net/TravianTools/

I'm going to be aiming at bringing everything together - between GetterTools & Killroid's Tools.

I'll be logging any information, new features, and idea lobbying in this thread - so if you have any concerns or requests, please post away.

Donkeh.

sounds interesting... but there are more pressing things that could be developed that dont exist yet, and its not hard to come up with something completely new rather then just copying a function that already exists.

If this is your first travian site, then Keep in mind that no part of your site can have ads, financials, or anyway for you to possibly benefit, otherwise you can expect dramas from the moderators, game admins, as well as agges. Be sure to read up on the Travian Fansite policy before you start.

Just so you know, you have an absolute uphill battle ahead of you, and every travian site owner that I have spoken to has regretted it. There are so many sites that I use (or used to use I should say) that have gone bust and the problem is getting bigger, and I think this is mainly due to how fast the travian playerbase is shrinking. TG announced on facebook just how severe their problems are and how many dead gameworlds are being closed... so its not really a good time to invest alot of time and effort into a dead niche when there are soo many other opportunities available.

Before you start. Treat this as a business idea.
1. do background research. Will your site really add something unique that isnt already offered?
2. do legal research, and read the travian fansite policy.
3. work out how much will the site cost to keep online per year
4. how many billable hours will you put into this project per year
5. How much will you spend on SEO per year in order to get traffic to your site? Without an SEO budget... no point starting the site at all. $1000-$2000 will probly do for the first year, and will get you on first page for alot of your keywords.
6. Ask yourself, if you are prepared to update the site weekly when travian changes things around again -> I have thousands of dollars worth of code that is useless. By the time its up to date and works, the next travian update breaks the code and you just get to the point where you have to just write off the loss and call it a day.

Donkeh
02 Aug 2013, 10:01 AM
Thank you for your interest, it's good to have all disadvantages listed and explained for me before I've gone too far into the project.

Basically this is going to be my own personal project - not a business idea, so I'm not interested in profiting wildly from this (in-line with fan-site policy).
My costs are minimal, i currently have my own dedicated server with unlimited everything - the only foreseeable cost would be a domain name in the future.
I used to work and complete SEO - a lot of it was no cost stuff - plus I'm not intending a crazy amount of traffic, it's more for my personal use & while I'm using it, i may as well give it away to others.

I want to bring everything that is out there together to save the long effort of having to copy+paste, fill in the boxes, etc. all the time - and make it look nice.
If i get it right, it'll basically be a one-stop-shop that is better, faster, easier to use, and better looking than the rest.
Having something 'brand new' will be a challenge - but i can throw ideas off a wall and come up with something eventually.
As for updates - that'll be the biggest hurdle, but good code is easily modified.

I guess the best thing for my project would be a community giving me a idea that they really want.

squibles
02 Aug 2013, 11:12 AM
Brilliant idea ... go for it

Donkeh
15 Aug 2013, 07:21 PM
I've done most of the back-end to the site. A lot of re-use-able-ready code to put into other sections.
The Troop page is half way done - just need to sort out the bottom half of the page (build time+upgrade time+upgrade efficiency).


If anyone wanted to contribute a page or something to my project - I've been using Perl (mods: DBI, CGI), and will be using heavy JavaScript for functionality, etc.

jawwwwsh
21 Aug 2013, 10:20 AM
May I pose a simple tool suggestion; A Trade Route creator. There used to be one at travian.aggenkeech.com but his site has been down for months. The site basically allowed you to input all parameters (Village target, resources, time, number of trips), and it would spit out links to preformed URLs so you can open all in tabs, and tab through clicking "save" and make 24 trade routes in about 45 seconds. I miss that thing so much! Edit - Also, there is a typo on the "gual" button on your troops tab, should say "gaul" :)

Donkeh
21 Aug 2013, 11:12 AM
Typo fixed
I'll see to doing a trade route script, it shouldn't take long. =]

septimus ii
21 Aug 2013, 12:12 PM
I'll poke the guy from aggenkeech to get his up again :D But yeah, that's a useful feature to have, until HQ get round to setting up the traderoutes properly :D

jawwwwsh
21 Aug 2013, 01:35 PM
Poke him!! POKE HIM WITH FURIOUS REPETITION!!! I NEED MY TRADEROUTE SITE.

Donkeh
23 Aug 2013, 09:38 PM
Trade Route script is made and up on my site!
Though there is one bug - you get a temp ban (1 second) if you use it - LOL :happy:

Emailed Admin - waiting for response to see if it i can fix it... And also to see about setting up other features that i believe to be edgy with the rules!
Otherwise best solution is to copy the link and paste into browser - use at own risk.

Bob Hawken
05 Sep 2013, 12:03 AM
Very interested to know how you get this sort of thing authorised for use.

I would like a project to get more into Javascript, Perl, etc and this area is ideal.

I thought that the issue was that I would not be able to access the required data easily and simply.

I have created a lot of Excel help tools for myself (which I believe to be within the rules as the info is cut and paste manually from the site) but if this could be automated any more that would be fantastic.

Thanks in advance to anyone who is able to enlighten me on this.

septimus ii
05 Sep 2013, 02:00 AM
I don't think you need authorisation as such. TG stipulates that you mustn't use adverts, and must credit them (the exact details will be in some set of rules somewhere :P )

But yeah, the tricky bit to getting started is accessing the publicly released server information. This (http://archive.forum.travian.co.uk/showthread.php?t=71601) is the thread that I've got bookmarked to study when i get round to trying something myself. I hope it's helpful.

Donkeh
05 Sep 2013, 09:14 AM
There is a little list of rules to follow to be a 'fansite' somewhere...
Though following these and not being in the 'bad-books' is similar to walking on egg shells.
I've already had an account deletion threat after querying Admin on a problem - my message was not read and instead keywords were picked and a generic response was given.
After repeat messages - I ended up just getting ignored. =[
So Authorized use is just a 'do and don't tell us' thing i speculate.

As for data - the map.sql is a given, daily update on the game play. You just need to download and run the SQL - it's very simple if you're putting it into a db.
Anything else of use can be found in Travian Answers, Travian Forums, and some on Travian Wikia.

Bob Hawken
05 Sep 2013, 10:03 AM
Is map.sql the only publicly available information? I know about this and download it into Excel to be able to mess about with it. I agree with Sam on another thread, so much easier than getting to grips with databases. She suggests that you would need a database to look at more than one day of data - I disagree. Excel 2007 and 2010 can emulate a relational database quite well with the amount of rows available, pivot tables and other tools if you know what you are doing.

Where does Getter get its information from?

Why are you allowed to use getter to interact with Travian? For example raiding from the active players or inactive players list...

For example, if I wanted to get data about sales prices in the market - is that possible or not (other than manually - which I have done for a couple of weeks on a server!)?

Samantha78
05 Sep 2013, 11:12 AM
I can do longer tonight; but the two ends of the spectrum in what is/isn't allowed is essentially that a person must push the buttons (hit ok, type into boxes etc.) not a script. It gets more complicated where some constructed urls pre-fill some of the boxes - the basic/public ones are fine; some of the more convoluted ones/if used extensively or very quickly seem to trigger the temp bans.

Map.sql is the only public and easy to extract data. Other information is public in game like the infra day rankings, but the only way to access on a large scale would break the rules/potentially damage performance.

Getter's information is all from map.sql (but it does rather more math/comparison on the file than most sites); it doesn't interact ie raid for you, but it does create the short cut links that that you to the send raid page with the village coords completed

septimus ii
05 Sep 2013, 01:55 PM
I've known people to get a temp ban without breaking any rules, so I don't think that's confirmation of breaking the rules. I think it just prevents you from carrying out an excessive number of actions per second, which is almost always due to a script.

I think the big advantage of using a DB over excel is that it's easier to automate the gathering and manipulating of the data, and that it's the only way to turn it into a website.

Bob Hawken
05 Sep 2013, 04:15 PM
You just made a powerful argument for me getting into DB.

Do you recommend Access? Or should I go straight to SQL? Or is there a better option?

Amazed that Getter runs off map.sql only.

septimus ii
05 Sep 2013, 05:56 PM
I reckon go straight to MySQL. This site (http://www.w3schools.com/sql/default.asp) has decent tutorials for a lot of coding stuff, but you can probably find something better :D

Samantha78
05 Sep 2013, 10:53 PM
What bit of getter surprises you? :)

Temp ban isn't necessarily breaking rules; but frequent temp bans probably is :)

Bob Hawken
05 Sep 2013, 11:16 PM
Well the links to the game really. Being able to raid lists of people quickly. I use it to mass-scout areas with one scout each. Can do at least 50 targets in under 5 minutes/

Not that I am complaining!

I get that the inactives is just comparing the database from one week to the next (or one day to the next). So can see that is achievable.

So in reality not that surprised.

Interested to know if anyone knows, why only the details on map.sql are available.

For instance it is possible to create a mapping of the whole of the server's 15c and 9c locations with their oases %s by doing a few cropfinder searches and collating the answers - I have an Excel heatmap that presents this quite nicely. This is the equivalent of looking at different areas of the map and finding out about different villages (in fact it is easier to do this for oases due to the cropfinder). The whole of the server is available for villages on map.sql but not the oases. Just find that sort of stuff strange.

No central datastore of all of the costs and times of different builds (real estate, fields and troops) - so useful and doesn't really take any significant effort as it is all available to Trav but collating this and being comfortable it has not changed from server to server is a bit of a pain.

I use that stuff to consider whether producing raiding troops is a better use of res than a particular level of field at different stages of the game. I also do something similar with the next build that I am going to do. Kirilloid is good for what the order is in one village but doesn't allow you to see what the best use of res is when you have multiple villages on the build.

Another thing I would like to be able to do would be to download the current state of all my villages (only) and I would have expected this to be available. I suppose I could write a parser like in getter for the troops (which is one thing that I find surprising that it is allowed) and do this but I don't really see the difference between doing this and having access to a download. The download may even be more efficient for Travian.

Sorry for the stream of consciousness.

Cheers.

Samantha78
05 Sep 2013, 11:33 PM
I did a crop heat map a few years ago too in excel :angel: (Show me yours? :p)

As far as I know, there has never been any development to map.sql ; it's simply the way it always has been (and I assume has never been a priority to change). (And gold is charged for access to the crop-finder, so I can see why that might not be disclosed information.)

I've never been entirely sure whether its fully appreciated the extent of data geeks that play the game...

Some of these things are super useful - potentially - but some undoubtedly carry a risk of being used to advantage cheats/bots by making information that can be used to control raiding/simming even more easily than is currently the case.

Bob Hawken
05 Sep 2013, 11:56 PM
I did a crop heat map a few years ago too in excel :angel: (Show me yours? :p)

As far as I know, there has never been any development to map.sql ; it's simply the way it always has been (and I assume has never been a priority to change). (And gold is charged for access to the crop-finder, so I can see why that might not be disclosed information.)

I've never been entirely sure whether its fully appreciated the extent of data geeks that play the game...

Some of these things are super useful - potentially - but some undoubtedly carry a risk of being used to advantage cheats/bots by making information that can be used to control raiding/simming even more easily than is currently the case.

Yes I get that... My view is however, that Trav should be up to the task of spotting botting. If they aren't then they need to employ someone who has previously been on the other side of the fence.

I am a massive data geek - some may have noticed. I will analyse this thing more than eat or sleep - the love of Excel and information and strategy in combo is fatal to me!

In terms of a crop heat map - mine is a 2 series bubble map in Excel. The two series are 15c and 9c. the size of the bubble shows the oases bonuses near that cropper. It isn't foolproof as there can be overlap of the oases which seem to give 3 15cs in one place - whereas in fact there is only 1 and 2 other fairly useless 15cs.

I'm happy to show you mine if you like - data for the current uk3 in one small area that I was interested in but easily upscaled for the whole server. Come to think of it, it would be really nice to merge it with map.sql to see which accounts are likely to be really dangerous - could easily identify hidden hammers...

How do you want me to reveal the goods??

PS cropfinder data file available to those who pay for the cropfinder? Simples?

Samantha78
06 Sep 2013, 12:22 AM
Some bots are easier to spot than others though - it is good to keep the developers on their toes tho too :) There is a wide variety out there.

oh, had better be skype lol; pm me :p I'm not interested in the real data lol...I'm interested in the design :D (I did it when trying to work out good places for groups of players to settle in the alliance together, and to sulk when one quad had way more/better 150% 15cs that others)

You will also want to look at the dynamic google graph in the s7 ww levels thread I think ;)

septimus ii
06 Sep 2013, 08:53 AM
I've done that just by making the spreadsheet into squares and colour-coding some of the cells to signify croppers and their wheat bonuses :D Not very elegant but it did the job. I did think of making a more interactive mapping tool, where you could take the map.sql and edit the map yourself, such as adding in croppers and oases, and adding in where your alliance members are going to settle. But the practicalities got the better of me :D

Bob Hawken
06 Sep 2013, 09:35 AM
Happy to show you the heat bubble map if you like. I liked it as the larger bubbles were for the bigger % croppers - I thought that made them stand out more than the cells solution - which I have done before as well. At the start of the server would be very useful to find the prime locations easily.

Another wish is to have the ability to search for 4536 (I am gaul) with at least 2 50% clay/wheat oases, or in fact any match up of a valley with a skew of res and the relevant 50% oases within the one of control of that square... Come to that 4437 with 100% wheat can be quite useful but currently no way to find them easily.

The practicalities of making an interactive one are not that hard. The ball breaker is getting the data in the first place and the inevitable de-duping to clean it up.

septimus ii
06 Sep 2013, 12:32 PM
Yeah, that was the issue :P Also security - I don't want any old person to see where my alliance members plan to settle. Straightforward to someone who's made a password protected website before, but I haven't :D

Donkeh
10 Sep 2013, 06:55 PM
Trade Route Planner is finished.
Two features -
1) Standard mode - normal options, able to put in multiple times seperated by commas.
2) Feeder moder - input pop, village type, factories, oasis, and also celebration info; town hall level, big party option. works out res per hour being used and remaining res to send to selected village.