DevelopmentVMHowto: Difference between revisions

From SoylentNews
Jump to navigation Jump to search
(Add more detailed instructions on applying the upgrades file, add info about slash having admin rights)
 
(7 intermediate revisions by one other user not shown)
Line 2: Line 2:


== Download and Install VM  ==
== Download and Install VM  ==
The Dev VM can be downloaded from http://torrents.soylentnews.org/.  This is a vim that is designed to run on VirtualBox. Information and downloads of VirtualBox are available here: http://torrents.soylentnews.org/.
The Slashdev VM can be downloaded from http://vm.soylentnews.org/Slashdev_14.10.ova.  This is the latest Slashdev VM and has many enhancements over the original version. It is a VM that is designed to run on VirtualBox. Information and downloads of VirtualBox are available here: https://www.virtualbox.org.


Once you get the VM downloaded and the VirtualBox up and ruining, you will want to do a '''File:Import Appliance''' in VirtualBox.  This will unpack the VM into you environment. Start up the VM and log in with the username and password of slash.
Once you get the VM downloaded and the VirtualBox up and running, you will want to do a '''File:Import Appliance''' in VirtualBox.  This will unpack the VM into you environment. Start up the VM and log in with the username and password of slash.  


The VM has port forwarding setup to you local system for ssh and http.   
The VM has port forwarding setup to you local system for ssh and http.   


<pre>ssh: 8022
<pre>ssh: 8022
http; 1337</pre>
http: 1337</pre>


For ssh your connection string should be:
For ssh your connection string should be:
Line 15: Line 15:
<pre>ssh slash@127.0.0.1 -p 8022</pre>
<pre>ssh slash@127.0.0.1 -p 8022</pre>


For websites, apache will use the name localhost instead of 127.0.0.1 so you will need to use:
User slash is in groups sudo and adm, so '''if''' you want to get a root shell you can get it by entering
 
<pre>sudo bash</pre>
 
Caution: Normally you won't need a root shell here.
 
For websites, Apache will use the name localhost instead of 127.0.0.1 so you will need to use:


<pre>http://localhost:1337</pre>
<pre>http://localhost:1337</pre>


Apache is turned off in the VM.  You will need to turn it on using the command:
Check if you can access external servers (will be required if you want to install additional packages or directly checkout from github). E.g.
 
<pre>ping soylentnews.org</pre>
<pre>apacheclt start</pre>
If the domain name cannot be resolved, you might want to shutdown the VM and add a bridged network adapter in network settings.


==Setup the Environment ==
==Setup the Environment ==
Line 31: Line 37:
cd ~/slashcode
cd ~/slashcode
git pull
git pull
</pre>
Now we need to install the new code to slash.  Here is a script that will help with that.  Create a new file called deployslash.sh in the slash home directory and chmod it to 755.
<pre>#! /bin/sh
echo === Install from git repo ===
echo
cd /srv/slashdev/slashcode
make USER=slash GROUP=slash SLASH_PREFIX=/srv/slashdev/slash install
echo
echo
echo === Clean up CSS and install Templates ==
echo
rm -rf /srv/slashdev/slash/site/slashdev/htdocs/*css
/srv/slashdev/slash/bin/symlink-tool -U
/srv/slashdev/slash/bin/template-tool -U
echo
echo
echo === Restart Apache==
echo
/srv/slashdev/apache/bin/apachectl restart
</pre>
</pre>


This script will be part of the next version of the Dev VM.
There are a couple of scripts set up in the vm that make deploying and reloading Slashcode easy.  The first script is deployslash.sh and the second is reloadslash.sh.  These scripts are in the shash user's home directory.  The deployslash.sh script will cd into the slashcode directory and install the Slashcode to the appropriate locations and then stop the slash daemon, stop and start Apache, clear the memcache, and finally restart the slash daemon. The reloadslash.sh script does the last steps only and is generally needed when updating templates, vars, and other items within the admin interface of the slash site.  Most changes will not take effect right away due to caching. The reload will empty the caches and unload any running Perl scripts from memory.


== MySQL Updates ==
== MySQL Updates ==
We need to update the dev environment to work with the new slashcode.  The update files are normally in /srv/slashdev/slashcode/sql/mysql/.  The upgrades file has the latest updates since that last major version of slashcode.  But here is a listing of the command that need to be run.  Save the code below to a file named upgrades.sql in your vm.
We sometimes need to update the Slashdev environment to work with the new Slashcode.  The update files are normally in /srv/slashdev/slashcode/sql/mysql/upgrades.  The upgrades file has the latest updates since that last major version of slashcode.  The 14.10 version of the VM is fully up-to-date with with the 14.10 release of Slashcode, so no updates to MySQL are currently needed.  You will need to add any future DB updates that are listed in the upgrades file to keep the new code working properly.
 
<pre>
USE slashdev;
 
# New SoylentNews Slashcode updates 2014-04-01.
 
# New variable for use with low submissions message.
INSERT INTO vars (name, value, description) VALUES ('subs_level','15','Level at which to not to display low submissions message, set to 0 to disable message');
 
# Update approvedtags_attr to add class to p and div.  For formatSub template changes.
UPDATE vars SET value = 'a:href_RU img:src_RU,alt_N,width,height,longdesc_U p:class div:class' where name = 'approvedtags_attr';
 
# Update menus table to remove users menu.  Needed for prefes update.
DELETE FROM menus WHERE menu='users';
 
# New variables for use disabling moderate_or_post
# NOTE: on production this is currently moderator_or_post
INSERT IGNORE INTO vars (name, value, description) VALUES ('moderate_or_post', '1', 'Can users moderate and post in the same discussion (1=yes, 0=no)');
 
# Add a missing variable to the database
INSERT IGNORE INTO vars (name, value, description) VALUES ('use_https_for_absolutedir_secure', '1', 'Should we use https as a secure absolutedir for nexuses (YOU PROBABLY WANT THIS!)');
 
# Once deleting the theme out, and installing the new "default" theme which is a giant catch-all, add a variable
# to control the default skin
ALTER TABLE users_info ADD COLUMN skin varchar(255) DEFAULT NULL;
INSERT INTO vars (name, value, description) VALUES ('default_skin','chillax','Default skin to use in-case the user has not selected one');
 
# New SoylentNews Slashcode updates 2014-06-01.
UPDATE vars SET value = 'slashcode_06_14' WHERE name = 'cvs_tag_currentcode';
 
# Allow submissions to have longer titles. story_text title is set to VARCHAR(100) so match here.
ALTER TABLE submissions MODIFY subj VARCHAR(100);
 
 
# Add new voting field
# NOTE: this shouldn't be in users, but it makes audioguys job easy, and the schema
# is already fucking ugly as sin so ....
ALTER TABLE users ADD COLUMN willing_to_vote tinyint UNSIGNED DEFAULT 0 NOT NULL;
 
 
# Disable portscanning by default
INSERT INTO vars (name, value, description) VALUES ('enable_portscan','0','Enable portscanning of proxys');
 
 
# Setup Imporved Threaded comments
INSERT INTO commentmodes (mode, name, description) VALUES ('improvedthreaded','Impoved Threaded','');
ALTER TABLE users_comments CHANGE mode mode ENUM('flat','nested','nocomment','thread','improvedthreaded') DEFAULT 'improvedthreaded' NOT NULL;
 
# Set all users on theraded or nested to improved threaded
UPDATE users_comments SET mode = 'improvedthreaded' WHERE mode = 'nested' OR mode = 'thread';
 
 
# Add in var to set if Newsletters are sent to new users by default
INSERT INTO vars (name, value, description) VALUES ('newsletter_by_default','0','Turn on newsletter sending by default for new users. 0 = off | 1 = on');
 
 
# Update for longer nicknames
UPDATE vars SET value = '35' WHERE name = 'nick_maxlen';
UPDATE vars SET value = '^[a-zA-Z_][ a-zA-Z0-9$_.+!*\'(),-]{0,34}$' WHERE name = 'nick_regex';
ALTER TABLE users CHANGE nickname nickname varchar(35) DEFAULT '' NOT NULL;
ALTER TABLE users CHANGE matchname matchname varchar(35);
 
 
# Add admin only for subscription prefs
INSERT INTO vars (name, value, description) VALUES ('subscribe_admin_only', 1, 'Subscription prefs only enabled for admins?');
 
 
# Remove deleted plugins
DELETE FROM site_info WHERE description LIKE '%Achievements%';
DELETE FROM site_info WHERE description LIKE '%PubKey%' or value = 'PubKey';
DELETE FROM site_info WHERE description LIKE '%Sphinx%';
DELETE FROM site_info WHERE description LIKE '%Users2%' or value = 'Users2';
DELETE FROM site_info WHERE description LIKE '%FireHose%';
DELETE FROM site_info WHERE description LIKE '%Tag%';
 
 
# Blocks update
ALTER TABLE blocks ADD COLUMN default_block tinyint UNSIGNED DEFAULT 0 NOT NULL;
ALTER TABLE blocks ADD COLUMN hidden tinyint UNSIGNED DEFAULT 0 NOT NULL;
ALTER TABLE blocks ADD COLUMN always_on tinyint UNSIGNED DEFAULT 0 NOT NULL;
UPDATE blocks SET default_block = 1 WHERE bid = 'poll' OR bid = 'index_more' OR bid = 'index_qlinks' OR bid = 'top10journals';
UPDATE blocks SET hidden = 1 WHERE bid = 'features_qlinks' OR bid ='recenttopics' OR bid ='spamurlregexes' OR bid ='subscriber_plug' OR bid ='articles_more' OR bid = 'features_more';
UPDATE blocks SET ordernum = 100 WHERE bid = 'mysite';
UPDATE blocks SET ordernum = 3 WHERE bid = 'topcomments';
UPDATE blocks SET seclev = 500 where bid = 'index_qlinks';
DELETE FROM blocks WHERE bid = 'newestthree';
 
# Remove all dynmaic user blocks or new blocks will not show up
TRUNCATE TABLE dynamic_users_blocks;
 
# Insert site news from old site_news template (Pull from your current site news rather than using this default if you already have it.)
INSERT INTO blocks (bid, block, seclev, type, description, skin, ordernum, title, portal, url, rdf, retrieve, default_block, always_on) VALUES ('site_news','<!-- begin site_news block -->\r\n\r\nYou should put some site news here that your users might enjoy.\r\n\r\n<!-- end site_news block -->\r\n\r\n',500,'static',NULL,'mainpage',1,'Site News',1,NULL,NULL,0,1,1);
 
# Delete slashboxes from all users
UPDATE users_index SET slashboxes = '';
 
 
# Delet story_never_author for all users to fix author bug (do this each time until bug fixed)
UPDATE users_index SET story_never_author = '';
 
 
# Allow Reltionship Change Messages to get sent for non subscribers
UPDATE message_codes SET subscribe=0 WHERE code=12;
 
</pre>


The commands must be run in mysql with root as the user and the database slashdev. Run the file above with the command:
MySQL is accessed with root as the user and the database slashdev. To access the DB use the command:
<pre>mysql -u root < upgrades.sql</pre>
<pre>mysql -u root slashdev</pre>


On the mysql prompt, to apply the upgrades file, enter
<pre>mysql&gt; source slashcode/sql/mysql/upgrades</pre>
== Update Slash ==
== Update Slash ==
Now lets update slash to our new code by running:
Now lets update slash to our new code by running:
Line 177: Line 56:
To do simple updates, you would make changes to the files in slashcode and redeploy using the deployslash.sh script.
To do simple updates, you would make changes to the files in slashcode and redeploy using the deployslash.sh script.


What if you want to develop code and have it merged into the main repo?  In this case you will need to setup your own copy of the repo do do your development work.  First if you do not have your own account on GitHub please set one up. Next go to the slashcode site https://github.com/SoylentNews/slashcode and click on Fork in the upper right.  Fork the repo to your user account.
What if you want to develop code and have it merged into the main repo?  In this case you will need to setup your own copy of the repo do do your development work.  First if you do not have your own account on GitHub please set one up. Next go to the Slashcode site https://github.com/SoylentNews/slashcode and click on Fork in the upper right.  Fork the repo to your user account.


Now on the VM go to slashcode.  We will setup your copy of the repo as a remote.  Run the commands:
Now on the VM go to slashcode.  We will setup your copy of the repo as a remote.  Run the commands:
Line 189: Line 68:


Once you get code you like, and have it committed to your personal fork, you can do a Pull Request, https://help.github.com/articles/using-pull-requests, that will create a ticket for the SoylentNews/slashcode repo to merge your commits to the master branch.  After proper testing by the Dev team and other sanity checks, you code will be merged and eventually put on to the production servers. See [[GitUse]] for more info about using git.
Once you get code you like, and have it committed to your personal fork, you can do a Pull Request, https://help.github.com/articles/using-pull-requests, that will create a ticket for the SoylentNews/slashcode repo to merge your commits to the master branch.  After proper testing by the Dev team and other sanity checks, you code will be merged and eventually put on to the production servers. See [[GitUse]] for more info about using git.


Just a note: I am new to git, so the above directions are what I have been using.  If people have a better method for doing this please feel free to update this page with the new information --pauulej72.
Just a note: I am new to git, so the above directions are what I have been using.  If people have a better method for doing this please feel free to update this page with the new information --pauulej72.


[[Category:Development]]
[[Category:Development]]

Latest revision as of 22:16, 18 February 2016

Development - parent

Download and Install VM

The Slashdev VM can be downloaded from http://vm.soylentnews.org/Slashdev_14.10.ova. This is the latest Slashdev VM and has many enhancements over the original version. It is a VM that is designed to run on VirtualBox. Information and downloads of VirtualBox are available here: https://www.virtualbox.org.

Once you get the VM downloaded and the VirtualBox up and running, you will want to do a File:Import Appliance in VirtualBox. This will unpack the VM into you environment. Start up the VM and log in with the username and password of slash.

The VM has port forwarding setup to you local system for ssh and http. 

ssh: 8022
http: 1337

For ssh your connection string should be: 

ssh slash@127.0.0.1 -p 8022

User slash is in groups sudo and adm, so if you want to get a root shell you can get it by entering 

sudo bash

Caution: Normally you won't need a root shell here.

For websites, Apache will use the name localhost instead of 127.0.0.1 so you will need to use: 

http://localhost:1337

Check if you can access external servers (will be required if you want to install additional packages or directly checkout from github). E.g. 

ping soylentnews.org

If the domain name cannot be resolved, you might want to shutdown the VM and add a bridged network adapter in network settings.

Setup the Environment

The slash user home directory is in /srv/slashdev/. The slashcode GitHub repository is located in slashcode. This is where you will make changes to the code you want to test. The actual slashcode install is located in slash.

The first thing you should probably do is update the repo to the latest version. 

cd ~/slashcode
git pull

There are a couple of scripts set up in the vm that make deploying and reloading Slashcode easy. The first script is deployslash.sh and the second is reloadslash.sh. These scripts are in the shash user's home directory. The deployslash.sh script will cd into the slashcode directory and install the Slashcode to the appropriate locations and then stop the slash daemon, stop and start Apache, clear the memcache, and finally restart the slash daemon. The reloadslash.sh script does the last steps only and is generally needed when updating templates, vars, and other items within the admin interface of the slash site. Most changes will not take effect right away due to caching. The reload will empty the caches and unload any running Perl scripts from memory.

MySQL Updates

We sometimes need to update the Slashdev environment to work with the new Slashcode. The update files are normally in /srv/slashdev/slashcode/sql/mysql/upgrades. The upgrades file has the latest updates since that last major version of slashcode. The 14.10 version of the VM is fully up-to-date with with the 14.10 release of Slashcode, so no updates to MySQL are currently needed. You will need to add any future DB updates that are listed in the upgrades file to keep the new code working properly.

MySQL is accessed with root as the user and the database slashdev. To access the DB use the command: 

mysql -u root slashdev

On the mysql prompt, to apply the upgrades file, enter 

mysql> source slashcode/sql/mysql/upgrades

Update Slash

Now lets update slash to our new code by running: 

deployslash.sh

Making Your Own Changes

To do simple updates, you would make changes to the files in slashcode and redeploy using the deployslash.sh script.

What if you want to develop code and have it merged into the main repo? In this case you will need to setup your own copy of the repo do do your development work. First if you do not have your own account on GitHub please set one up. Next go to the Slashcode site https://github.com/SoylentNews/slashcode and click on Fork in the upper right. Fork the repo to your user account.

Now on the VM go to slashcode. We will setup your copy of the repo as a remote. Run the commands: 

git remote add {name} {url-to-your-GitHub-repo}
git fetch {name}
git checkout {name}/master

Redeploy slash and you are working from your own repo. As you may have noticed the primary work is being done on the master branch. You will not need to worry about the other branches of the code for now.

Once you get code you like, and have it committed to your personal fork, you can do a Pull Request, https://help.github.com/articles/using-pull-requests, that will create a ticket for the SoylentNews/slashcode repo to merge your commits to the master branch. After proper testing by the Dev team and other sanity checks, you code will be merged and eventually put on to the production servers. See GitUse for more info about using git.

Just a note: I am new to git, so the above directions are what I have been using. If people have a better method for doing this please feel free to update this page with the new information --pauulej72.

Retrieved from "https://wiki.staging.soylentnews.org/index.php?title=DevelopmentVMHowto&oldid=9721"