EOSERV Forum > EOSERV > Tutorial: Automatically convert pub files to CSV.
Page: << 1 >>
Tutorial: Automatically convert pub files to CSV.
Author Message
Post #202769 Tutorial: Automatically convert pub files to CSV.

If you are interested in documenting your server data without the use of WebCP, one approach might be to convert the pub data to an CSV file and import it into Google Sheets.

I created a system to automatically convert the pub files into CSV files each time the server is loaded and on $repub.

Tutorial below:


In one of the config ini files, add the following:
## CSVSaveECF, CSVSaveEIF, CSVSaveESF, CSVSaveENF (string)
# Paths to save pub files in CSV format.
# If path is not empty, it will save the corresponding pub file to a comma-separated values (CSV) file.
# This action is performed every server boot and $repub command.
# This may be useful for documenting the data files into a spreadsheet format.
# You can convert CSV files to Google Sheets: https://www.google.com/sheets/about/
CSVSaveECF = ./data/pub/dat001.ecf.csv
CSVSaveEIF = ./data/pub/dat001.eif.csv
CSVSaveESF = ./data/pub/dsl001.esf.csv
CSVSaveENF = ./data/pub/dtn001.enf.csv

## CSVShowLog (bool)
# Will output a successful message to the console or output file, if CSV data was saved.
# See: CSVSaveECF, CSVSaveEIF, CSVSaveESF, CSVSaveENF (string)
CSVShowLog = yes

In eoserv_config.cpp "eoserv_config_validate_config()" method, add the following:


    eoserv_config_default(config, "CSVSaveECF""");
    eoserv_config_default(config, "CSVSaveEIF""");
    eoserv_config_default(config, "CSVSaveESF""");
    eoserv_config_default(config, "CSVSaveENF""");
    eoserv_config_default(config, "CSVShowLog"false);

In world.cpp replace both occurrences of the following: (There's 2 occurences)
    this->eif->Read(this->config["EIF"]);
    this->enf->Read(this->config["ENF"]);
    this->esf->Read(this->config["ESF"]);
    this->ecf->Read(this->config["ECF"]);

To:

    this->eif->Read(this->config["EIF"], this->config);
    this->enf->Read(this->config["ENF"], this->config);
    this->esf->Read(this->config["ESF"], this->config);
    this->ecf->Read(this->config["ECF"], this->config);

In eodata.hpp, add:
#include "config.hpp"

Find: (around line #186)

        EIF(const std::string& filename) { Read(filename.c_str()); }

        void Read(const std::string& filename);

Replace with:

        EIF(const std::string& filename, Config eoserv_config) { Read(filename.c_str(), eoserv_config); }

        void Read(const std::string& filename, Config eoserv_config);

Find: (around line #260)

        ENF(const std::string& filename) { Read(filename.c_str()); }

        void Read(const std::string& filename);

Replace:

        ENF(const std::string& filename, Config eoserv_config) { Read(filename.c_str(), eoserv_config); }

        void Read(const std::string& filename, Config eoserv_config);

Find: (around line #336)

        ESF(const std::string& filename) { Read(filename.c_str()); }

        void Read(const std::string& filename);

Replace:

        ESF(const std::string& filename, Config eoserv_config) { Read(filename.c_str(), eoserv_config); }

        void Read(const std::string& filename, Config eoserv_config);

Find: (around line #381)

        ECF(const std::string& filename) { Read(filename.c_str()); }

        void Read(const std::string& filename);

Replace:

        ECF(const std::string& filename, Config eoserv_config) { Read(filename.c_str(), eoserv_config); }

        void Read(const std::string& filename, Config eoserv_config);

In eodata.cpp:

Add this method:

static std::FILE* eodata_file_open_w(const std::string filename) {
    return !filename.empty() ? std::fopen(filename.c_str(), "w") : NULL;
}

Replace "EIF::Read()" method with:

http://pastebin.com/25N30GRw

Replace "ENF::Read()" method with:

http://pastebin.com/4XFQ6HgN

Replace "ESF::Read()" method with:

http://pastebin.com/whUqMN7j

Replace "ECF::Read()" method with:

http://pastebin.com/qn9DXekA


That's it. You can now set the location it saves the csv files. Or leave the values blank to disable them.

Example:

CSVSaveECF = ./data/pub/dat001.ecf.csv
CSVSaveEIF = ./data/pub/dat001.eif.csv
CSVSaveESF = ./data/pub/dsl001.esf.csv
CSVSaveENF = ./data/pub/dtn001.enf.csv

You can also disable the logging by setting the following to false:

CSVShowLog = no

Example of what the CSV looks like when imported to Google Sheets:







---
Just your friendly neighborhood Programmer-Man!
7 years, 7 weeks ago
Post #202772 Re: Tutorial: Automatically convert pub files to CSV.

Very nice! Now can you edit it in google sheets and convert back into pub format? Imagine how powerful it would be if you could edit a pub file in a text format using an advanced editor and use things such as find and replace, highlight, etc.

I remember when I was 12 and playing with my first private servers the pub files kept getting damaged or, one time I used a malware that said it would pack all my clothes for me *cough I know who you are* on my production pub without a backup, and I was pretty much screwed and had to redo my entire item pub.. but it was okay I had a good memory back then and was able to fix it within 2 hours.

Anyways, once you're able to export to a text format you can definitely use version control with the pub files, (which I hope you're already doing with your source files). And that can be a really great asset if you start working with a team of people. Of course the server needs a way to read the text format or there has to be a way to convert back.

---
I not hacker

“Everybody is a genius. But if you judge a fish by its ability to climb a tree, it will live its
whole life believing that it is stupid.” - Albert Einstein : Really Great Quote Ramy!
7 years, 7 weeks ago
Post #202775 Re: Tutorial: Automatically convert pub files to CSV.

We actually are thinking about doing just that except it doesn't hold our current focus just yet : )!

---
Graphics Designer.
"I'll keep believing in the future, not caring if anyone laughs at me"
Hi : D I am me!
7 years, 7 weeks ago
Post #202776 Re: Tutorial: Automatically convert pub files to CSV.
Hacker_Alex posted: (9th Mar 2017, 03:52 pm)

Very nice! Now can you edit it in google sheets and convert back into pub format? Imagine how powerful it would be if you could edit a pub file in a text format using an advanced editor and use things such as find and replace, highlight, etc.

I remember when I was 12 and playing with my first private servers the pub files kept getting damaged or, one time I used a malware that said it would pack all my clothes for me *cough I know who you are* on my production pub without a backup, and I was pretty much screwed and had to redo my entire item pub.. but it was okay I had a good memory back then and was able to fix it within 2 hours.

Anyways, once you're able to export to a text format you can definitely use version control with the pub files, (which I hope you're already doing with your source files). And that can be a really great asset if you start working with a team of people. Of course the server needs a way to read the text format or there has to be a way to convert back.


Thank you. Converting back to a pub file would indeed be powerful. As of now, I don't think I have the required knowledge to read and write the raw data of the pub files. I would love to be able to edit and write to pub files in real-time though. It would have so many benefits. One example would be to append an automatic level to the mob names, then save it to a second version of the pub file, and then send that edited pub file to the clients (instead of the original pub file). Another example is what you spoke about: converting a CSV file back to a pub file.

Like you said, I could also read the plain text (csv) data and parse it on the server, instead of the pub file format. But I assume it would have its own downside. I also think a custom client would be needed to also read the plain text format for the pub files (which also means the user could edit the pub files easily).

That must have been such a pain to redo the entire pub file. We are using version control, I can't imagine going back to working on projects without version control.

---
Just your friendly neighborhood Programmer-Man!
7 years, 7 weeks ago
Post #202783 Re: Tutorial: Automatically convert pub files to CSV.

I had considered outputting to xml or html for cheap Web page building. I had never considered using csv as a middle man pub editor. Essentially that is a bit of extra work to avoid writing some dialog boxes in win32 api so you can use something like MS Excel to edit pub files. Anyway, the cool thing about this is that it can be used to see multiple entries of data at once. 

7 years, 7 weeks ago
Page: << 1 >>

EOSERV Forum > EOSERV > Tutorial: Automatically convert pub files to CSV.