AIC Group Project Big Data
Architecture: https://www.dropbox.com/s/lqbvh8k3ydv8mqv/AIC_Architektur_Vorschlag.pdf?dl=0
- Gradle 2+ (http://www.gradle.org)
- Java 7+
- MongoDB (http://www.mongodb.org/downloads & http://docs.mongodb.org/manual/)
gradle build // default build task
gradle startStream // start stream and write to mongodb/sql
gradle analyze // extract graph from mongodb/sql
gradle userConvert // convert existing Users from mognodb into H2 Database
gradle userClean // clean and recreate H2 user table
gradle appRun // starts WebApplication
mongo.properties // configure MongoDB & Collection names
server.properties // configure streaming options (terms,inital users, ...)
neo4j.properties // configure Neo4j location
sql.properties // configure H2 DB location
twitter.properties // twitter credentials
Neo4J: /data/neo/aicDB
SQL: /data/sql/userDB
- install software
- get source
- "gradle build" should build without problems
- "gradle startStream" will collect data (skip this step if you already have a dataset in your mongodb and users in the H2 database), task need to be stopped manually.
- "gradle analyze" extract the graph from the databases. (skip this step if you already have the extracted neo4j database(e.g. VM) )
- "gradle appRun" starts the web server and explore the collected data and queries.
- developers will provide you with the password = 'aic-bigdata' *
- open terminal and change directory to ~/aic-bigdata/aic-bigdata-server
- execute "sudo gradle appRun"
- connect with your browser to http://localhost:8080/aic-bigdata-server
corrupt Neo4j:
- if your Neo4j database is corrupt please delete the folder /data/neo/aicDB
- execute "sudo gradle analyze" in the "aic-bigdata-server" directory and the neo4j should be restored.
- Search and filtering should work case insensitive, if not (H2 varchar_ignorecase is not set -> gradle userClean to recreate the user table).
- Dont forget to analyze your data :
-