Page History

# [Overview|#overview]
# [Producer|#producer]
# [Consumer|#consumer]
# [Repository|#repo]
# [Data Model|#datamodel]
# [Searching in NOCLook|#search]
# [Indexed properties|#index]
# [Backup and Restore|#backuprestore]
# [Links|#links]
# [How to set up a NI suite|#setup]


{anchor:overview}
h3. Overview
NORDUnet Network inventory project is divided in to three parts. One part that gathers the data (producer), another part that uses the data (consumer) and a repository for the raw data.

{anchor:producer}
h3. Producer
For the producer we use the [NERDS projects|https://github.com/fredrikt/nerds] JSON format to save the data. How to actually collect data depends on what you want to collect data from.

{anchor:consumer}
h3. Consumer
The only NORDUnet consumer today is [NOCLook|http://git.nordu.net/?p=norduni.git;a=shortlog;h=refs/heads/operation] which is a web gui made with the Django web framework and Neo4j graph database as back end for the [inventory data model|https://portal.nordu.net/display/nordunetops/Network+Inventory+Project#NetworkInventoryProject-pdatamodel]. The data is added to the Neo4j database via the REST interface using [norduni_client.py|http://git.nordu.net/?p=norduni.git;a=history;f=src/niweb/norduni_client.py] which builds on [neo4j-rest-client|https://github.com/versae/neo4j-rest-client].

Neo Tech has release official [python bindings|http://pypi.python.org/pypi/neo4j-embedded] and they are used in the [master branch|http://git.nordu.net/?p=norduni.git;a=summary] of NOCLook. The speed improvement is great so they will be used in the operational NOCLook in a near future.

{anchor:repo}
h3. Repository
[Git|http://git-scm.com/] is used to handle the revisions of the collected data.

{anchor:datamodel}
h3. Data model

!NI_Data_Model.png|thumbnail!
Hopefully these entities and relationships are enough.

!NI_Prototype_Data.png|thumbnail!
The data that was inserted manually in the prototype.

{anchor:search}
h3. Searching in NOCLook
In addition to the search box at the top of the page you can also use the URL field for searching.

{code}
/findall/[value]/
ex. /findall/10/
/findall/[key]/[value]/
ex. /findall/link/10/
/findin/[node type]/[value]/
ex. /findin/optical-node/10/
/findin/[node type]/[key]/[value]/
ex. /findin/optical-node/link/10/
/getall/[node type]/
ex. /getall/optical-node/
{code}

{anchor:index}
h3. Indexed properties
Neo4j provides [Lucene indexes|http://lucene.apache.org/java/docs/index.html] for nodes and relationships. A "search" index is created with the properties that will be searched when a user uses the search bar. Some optimization indexes was also created to speed up the consuming or presentation.

*Node search index:*
name (all nodes)
description (interface and unit nodes)
ip_addresses (unit and host nodes)
as_number (peering partner nodes)
hostname (host nodes)

*Relationship search index:*
ip_address (Uses and Depends_on relationships)

*Node node_types index*
node_type (all nodes)

All property values above is also indexed under the index key *all* in all indexes. For example in the search index you can query the all-key for both name, description, ip_addresses, as_number and hostname.

{anchor:backuprestore}
h3. Backup and Restore
You can backup the Postgresql and Neo4j databases using these scripts:
{code}
# Backup the SQL database
su - postgres -c "pg_dumpall > /opt/nistore/producers/noclook/sql/postgres.sql"
# Run NOCLook producer for backup purpose
cd /opt/norduni/src/scripts/
rm /opt/nistore/producers/noclook/json/*
./noclook_producer.py -O /opt/nistore/producers/noclook/json/
# Push the changes to nistore master
/usr/local/sbin/ni-push.sh -r /opt/nistore/
{code} 

{code|title:

}
# All producers need to be listed here with a path data or empty
[data]
juniper_conf = 
nmap_services = 
alcatel_isis = 
noclook = /opt/nistore/producers/noclook/json/
{code}