tag:blogger.com,1999:blog-25976732637255945792024-03-13T03:51:31.328+02:00Dikran's blog - "Research, Create, Share"About Java, Web and related technologiesDikran Seropianhttp://www.blogger.com/profile/18440451836835726018noreply@blogger.comBlogger10125tag:blogger.com,1999:blog-2597673263725594579.post-7413967447985723152021-02-08T17:23:00.013+02:002021-09-16T14:32:35.515+03:00SLF4j Logging performance: lazy argument evaluation<script src="https://cdn.rawgit.com/google/code-prettify/master/loader/run_prettify.js"></script>
<style>
.prettyprint {
background: white;
font-family: Monaco, Consolas, Courier New, monospace;
font-size: 12px;
line-height: 1.5;
border: 1px solid #ccc;
padding: 10px;
word-wrap: initial;
overflow-x: scroll;
}
</style>
<p>
Sometimes we need to log a dynamically generated expressions that are very
expensive to compute. For instance I had to log an object to yaml format only
when debug was enabled. Serializing an object to yaml is an expensive
operation especially when you need to scale up to thousands of calls per
second.
</p>
<p>(As reference, I am using java 8 with slf4j-1.7.25)</p>
<p>If I had directly used</p>
<pre class="prettyprint lang-java">
log.debug("The message is:{}", toYamlString(myObject));
</pre>
<p>
then the message generating method would be called every time even if debug as
disabled on the logger. This is because of the <a href="https://docs.oracle.com/javase/specs/jls/se11/html/jls-15.html#jls-15.12.4">java argument evaluation</a> mechanism.
</p>
<p>The obvious choice here is:</p>
<pre class="prettyprint lang-java">
if(log.isDebugEnabled()){
log.debug("The message is:{}", toYamlString(myObject));
}
</pre>
<p>but, apart of adding unpleasing code on top of your method, this is also doubling
the call on if(log.isDebugEnabled()) that is also performed in the logging
framework itself.So I took some time to see if it could be done in a better way.</p>
<p>At some point I found <a href="https://itnext.io/lazy-logging-40314cf9bb25">this post</a> that was nicely solving this. I liked it and
wrote my code accordingly. Then I realised it could be even simpler!</p>
<p>So I simplified it to only this:</p>
<pre class="prettyprint lang-java">
private static Object lazyString(final Supplier<?> stringSupplier) {
return new Object() {
@Override
public String toString() {
return String.valueOf(stringSupplier.get());
}
}
</pre>
<p>Then in my logging call:</p>
<pre class="prettyprint lang-java">
log.debug("The message is:{}",lazyString(() -> toYamlString(myObject)));
</pre>
<p>or, if your method take no arguments, you can use method reference:</p>
<pre class="prettyprint lang-java">
log.debug("The message is:{}",lazyString(this::toYamlString));
</pre>
<p>That's it! Simple and elegant.</p>
<p>
The good news is that more and more logging frameworks added or are adding
native support for deferred evaluation of arguments.
</p>
<p>Until then we can use simple nice workarounds like this.</p>
<p>Have a nice day,</p>
<p></p>
<p>Dikran.</p>
Dikran Seropianhttp://www.blogger.com/profile/18440451836835726018noreply@blogger.com0tag:blogger.com,1999:blog-2597673263725594579.post-19721597376732546242018-04-03T13:01:00.001+03:002019-01-14T14:30:28.536+02:00Compile Maven project and tests with different compilers and with different unit and integration test directoriesMy project is java, and I wanted to give my team the possibility to use java/junit and groovy/spock for our tests.<br />
<br />
Moreover I wanted to keep unit tests separated from integration tests and if possible with different compilation life cycles so that the flow is:<br />
<br />
1. compile the project code from src/main/java using the default compiler<br />
2. compile and run the unit tests using the mixed java-groovy eclipse compiler<br />
3. compile and run the integration tests using the mixed java-groovy eclipse compiler<br />
<br />
This way the production code is compiled natively while we can play with java-groovy mixed classes in unit and integration tests.<br />
<br />
After digging a lot and trying many unsuccessful approaches I got it working exactly as I wished.
<br />
Here is the pom:<br />
<br />
<script src="https://gist.github.com/seropian/36a49fedc3fcdf349e632995661909db.js"></script>
As you may notice, I have left the unit tests in the standard maven path ie. <code>src/test/java</code>, but, if I want to further move them to <code>src/test/unit/java</code> then I need to configure both the compiler section and the surefire-plugin section in the same way I did for the integration tests.<br />
<br />
Basically the secret in in instructing the compiler on:<br />
- when to run (we configure this aspect within an <code>execution</code> section)<br />
- where to compile sources from - within the element <code>compilesourceroots</code><br />
- where to output classes<br />
- what classes (by name or pattern) to include - in the element <code>outputDirectory</code><br />
and at the same time to instruct the test runner (surefire or failsafe) on:<br />
- where the test sources are located - within <code>testSourceDirectory</code><br />
- when the test classes are located - within <code>testClassesDirectory</code>
<br />
<br />
One essential thing to notice is the id of each execution element (in our case <code>default-testCompile</code> and <code>integration-testCompile</code>, because maven identifies each instance by it's id s it must be uniquely named.<br />
<br />
Another thing that many don't know is that the ids can be overridden and indeed I have used the default maven compiler id for the unit test compilation so that only the eclipse compiler shall be run instead of runing also the default compiler. you can change the id and test yourself.<br />
<br />
Hope that shall help you too!<br />
<br />
Cheers,
DikranDikran Seropianhttp://www.blogger.com/profile/18440451836835726018noreply@blogger.com1tag:blogger.com,1999:blog-2597673263725594579.post-34637400785582651772016-10-14T17:43:00.005+03:002016-10-21T16:55:58.191+03:00How to install docker on centos 6 - quick and dirty<h2>
Installing docker 1.12 on centos 6.8</h2>
While doing consultancy work at one big telecom company, I proposed introducing Docker in the process of automating and autoscaling parts of the software infrastructure, especially on the the java development chain and runtime deployment sides. They were enthusiastic about this so I got the task of making it happen. Just that at the time I did not know their infrastructure (all Centos 6.8) did not support docker...<br />
<br />
Who got here must be quite desperate, as I got for a while after taking this task.<br />
Lots of research and trial got me to put together the following instructions that made it work.<br />
<br />
This is an unpolished, quick hand log of what I had to do in order to make docker successfully run on this OS version.<br />
<br />
<b>WARNING:</b> Be sure of your deep linux understanding and knowledge before trying this into production systems!
<br />
<script src="https://gist.github.com/seropian/86294e209dae3d4774846ccd4d9a6c6c.js" type="text/javascript"></script>
<br />
Hopefully it shall help. I'll come back to this post time allowing, to better arrange, document and cleanup.<br />
<br />
Cheers,<br />
Dikran<br />Dikran Seropianhttp://www.blogger.com/profile/18440451836835726018noreply@blogger.com0tag:blogger.com,1999:blog-2597673263725594579.post-63946028867009710012016-03-30T12:17:00.001+03:002019-01-14T14:32:19.748+02:00Updating all branches from all local git projects in one shot<script src="https://cdn.rawgit.com/google/code-prettify/master/loader/run_prettify.js"></script>
<style>
.prettyprint {
background: white;
font-family: Monaco, Consolas, Courier New, monospace;
font-size: 12px;
line-height: 1.5;
border: 1px solid #ccc;
padding: 10px;
word-wrap: initial;
overflow-x: scroll;
}
</style>
<br />
<div>
There are many times when I need to update at once more than just one git project.<br />
I usually structure my projects under a common directory like /Users/Dikran/workspace/projects.<br />
When I update I <code>cd</code> to the respective project and <code>git pull</code>.<br />
<br />
I justs happens that recently I needed two things:<br />
1) check updated code of more than one project.<br />
2) check changes made on other branches than the current one.<br />
<br />
As you know, git pull is updating only the current branch in a project. Moreover it has the following limitations (quoting from git-up project):<br />
<br />
<code>
"It merges upstream changes by default, when it's really more polite to rebase over them, unless your collaborators enjoy a commit graph that looks like bedhead.<br />
It only updates the branch you're currently on, which means git push will shout at you for being behind on branches you don't particularly care about right now."
</code>
<br />
<br />
So in order to solve those needs at once, there is a simple solution enabled by a simple script and a great git extension called <code>git-up</code>. This is a very convenient tool that does many nice things in completion to what git already offers. Check <a href="https://github.com/aanand/git-up">the site</a> for docs and info.</div>
<br />
<div>
The steps:<br />
<br />
1. Install git-up extension</div>
<div>
For Ruby (the original): <br />
<pre class="prettyprint lang-bash">gem install git-up</pre>
Or for the Python port: <br />
<pre class="prettyprint lang-bash">pip install git-up</pre>
<br />
2. Create a script (i.e. <code>updateAll.sh</code>) in the root directory of your git subprojects, containing the following:<br />
<pre class="prettyprint lang-bash">#!/bin/bash
set -x
for project in */
do git -C $project up &
done
wait
</pre>
The script cycles through all subdirectories in the current directory, and issues background calls to git-up on every discovered directory.<br />
<br />
The <code>wait</code> is added at the end so that the script shall exit only after all directory updating commands finished.<br />
<br />
For a variation you can filter directory names if for instance you want updated only specific directories within a certain project. So for instance if you have your main project called <code>myshop</code> and the composing modules are <code>myshop-frontend/ myshop-backend/ myshop-tests/</code> then just change the
<br />
<pre class="prettyprint lang-bash">for project in */</pre>
with
<br />
<pre class="prettyprint lang-bash">for project in myshop*/</pre>
</div>
<div>
Thats't all. Simple, isn't it?<br />
<br />
A warning note though. Although git-up suits most cases, please check the documentation first, to be sure it won't mess things in your specific project's commit conventions.</div>
<br />
<div>
Have a nice day,<br />
Dikran
</div>
Dikran Seropianhttp://www.blogger.com/profile/18440451836835726018noreply@blogger.com0tag:blogger.com,1999:blog-2597673263725594579.post-38221849842192840432015-12-01T22:28:00.002+02:002019-01-14T14:33:50.842+02:00Automated Testing Aid: Manually Running a Quartz Job<script src="https://cdn.rawgit.com/google/code-prettify/master/loader/run_prettify.js"></script>
<style>
.prettyprint {
background: white;
font-family: Monaco, Consolas, Courier New, monospace;
font-size: 12px;
line-height: 1.5;
border: 1px solid #ccc;
padding: 10px;
word-wrap: initial;
overflow-x: scroll;
}
</style>
I work in a project where testing is a first class citizen. We do unit tests, security tests, integration tests, end-to-ends api tests (SBE), and end-to-end functional (interface based) tests also by using SBE.<br />
All right, everything's fine, until I need to throw in some asserts at the end of my integration/sbe test where I should check if the whole process performed well. Ok, only that this part of the process is accomplished by quartz jobs that run asynchronously in their specific setup, beyond our control.
<br /><br />
<i><b>Note</b>: The assumption for this post is that your Quartz scheduler can be run within the same application with your tested classes. For distributed quartz jobs there is another story.</i>
<br /><br />
Some could say, ok, by you have the possibility to get a job by it's name and call <code>triggerJob(JobKey)</code> on a quartz scheduler instance, that should trigger the job immediately. But, be careful is about triggering a job, and not running the job. That means that:
<br />
<br />
<ul>
<li>the command is asynchronous and return immediately; </li>
<li>the job could actually start later, depending on the schedule's config and </li>
<li>you don't actually know when the job shall finish so that you can test your assumptions about the outcome of it.</li>
</ul>
<br />
Two quick solutions:<br />
<ol>
<li>after test's execution finished, before asserting on data, sleep the test thread for a while to give quartz time to do it's work. But sleep for how long? Some manual tries could give us some empirical idea of how long should we wait before the jos is usually executed, but we are never going to be 100% sure it actually was. And then, this approach could bring the execution of our test suite to last forever, imagine running hundreds of tests of this type that each are sleeping for few seconds... It doesn't sound very appealing.</li>
<li>add a JobListener listener to the scheduler, then trigger the job and then put your main thread in wait until the listener is triggered on execution finished, and notifies your main thread so it can resume it's testing task. But, again, there might be many jobs already triggered and running until ours get's it's change to run. And after all, would you really want to get into unexpected threading issues? I think not.</li>
</ol>
<br />
So, after trying the aforementioned approaches and not really being happy with them I thought, why not directly run the jobs I am directly interested in? <br />
Well this is not that trivial, because I'd like to run the jobs as they are, without having to know what other stuff is injected in each of my classes extending <code>QuartzJob</code> in order to make it work. So, after some research and study of how quartz works in collaboration with spring, that is what came out:<br /><br />
<br />
<pre class="prettyprint lang-java">import org.quartz.Job;
import org.quartz.JobExecutionContext;
import org.quartz.Scheduler;
import org.springframework.beans.BeanWrapper;
import org.springframework.beans.BeansException;
import org.springframework.beans.MutablePropertyValues;
import org.springframework.beans.PropertyAccessorFactory;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationContextAware;
import java.lang.reflect.Method;
import java.util.Map;
public class ManualJobExecutor implements ApplicationContextAware {
private ApplicationContext applicationContext;
public void executeJob(final Class<Job> jobClass) {
try {
//create job instance
final Job quartzJob = jobClass.newInstance();
// For the created job instance, search all services that are injected by quartz.
// Those service instances are kept inside each scheduler context as a map
final BeanWrapper beanWrapper = PropertyAccessorFactory.forBeanPropertyAccess(quartzJob);
final MutablePropertyValues propertyValues = new MutablePropertyValues();
//get all schedulers defined across all spring configurations for this application
final Map<String, Scheduler> schedulers = applicationContext.getBeansOfType(Scheduler.class);
for (final Scheduler scheduler : schedulers.values()) {
// Populate the possible properties with service instances found
propertyValues.addPropertyValues(scheduler.getContext());
}
//set the properties of the job (injected dependencies) with the matching services
//the other services in the list that have no matching properties shall be ignored
beanWrapper.setPropertyValues(propertyValues, true);
//get method executeInternal(JobExecutionContext) from job class extending QuartzJobBean
final Method executeJobMethod = quartzJob.getClass().getDeclaredMethod("executeInternal", (JobExecutionContext.class));
executeJobMethod.setAccessible(true);
//call the processItems method on the Job class instance
executeJobMethod.invoke(quartzJob);
} catch (final Exception e) {
throw new RuntimeException(String.format("Exception while retrieving and executing job for name=%s", jobClass.getName()), e);
}
}
@Override
public void setApplicationContext(final ApplicationContext applicationContext) throws BeansException {
this.applicationContext = applicationContext;
}
}
</pre>
<br />
That's it!<br />
Of course there are also other aspects, i.e checking if other job of the same class is already executing so that it won't overlap with your execution. Usually in Quarz, <code>@DisableConcurrentExecution</code> takes care of this but here you need to check it yourself.<br />
You could also make your method accept a job by its name instead of class so you can get the names from your database instead of looking into project classes.<br /><br />
I hope this is going to ease your testing.<br />
Please share your thoughts.<br />
<br /><br />
Have a nice day,<br />
Dikran
Dikran Seropianhttp://www.blogger.com/profile/18440451836835726018noreply@blogger.com1tag:blogger.com,1999:blog-2597673263725594579.post-11334750897757445582015-05-27T19:29:00.000+03:002019-01-14T14:34:38.454+02:00Spring Boot & Jasypt easy: Keep your sensitive properties encrypted<script src="https://cdn.rawgit.com/google/code-prettify/master/loader/run_prettify.js"></script>
<style>
.prettyprint {
background: white;
font-family: Monaco, Consolas, Courier New, monospace;
font-size: 12px;
line-height: 1.5;
border: 1px solid #ccc;
padding: 10px;
}
code {
font-family: Monaco, Consolas, Courier New, monospace;
font-size:12px;
}
</style>
<br />
<h3>
Goal</h3>
<br />
I want to store my database password encrypted in the application properties file and provide the property encryption password at runtime as java system property or environment variable.<br />
<br />
<h3>
Context:</h3>
<br />
Java 7, Spring Boot 1.2.3.RELEASE<br />
Currently Spring Boot does not offer native property encryption support.<br />
<br />
<h3>
Solution</h3>
<br />
Use <a href="http://www.jasypt.org/" rel="nofollow" target="_blank">jasypt </a>encryption library and integrate it into Spring Boot's configuration flow.<br />
<br />
How?<br />
Here is a quick and dirty example:<br />
<br />
1. <a href="http://www.jasypt.org/download.html" rel="nofollow" target="_blank">Download </a>jasypt and unzip the contents in a folder;<br />
2. Choose a password for encrypting your sensitive properties; for the purpose of this example we choose "my-encryption-password";<br />
3. Choose the property you want encrypted; here we choose to encrypt the database password "my-database-password";<br />
4. Encrypt the database password ("my-database-password") using jasypt and the encryption password ("my-encryption-password"); go into the jasypt bin folder and run:<br />
<br />
<pre class="prettyprint lang-bsh">$ encrypt.sh input=my-database-password password=my-encryption-password
----ENVIRONMENT-----------------
Runtime: Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 24.60-b09
----ARGUMENTS-------------------
input: my-database-password
password: my-encryption-password
----OUTPUT----------------------
TJ1vA+DLWFrwEmbZKmGmawEonbJw4DxhkFf53JzKfvY=
</pre>
<br />
The output is the encrypted password.<br />
To configure the database in the SpringBoot's application.properties we add:<br />
<br />
<pre class="prettyprint">#for this example we use H2 database
spring.datasource.driver-class-name=org.h2.Driver
spring.datasource.url=jdbc:h2:mem:my-schema
spring.datasource.username=test-user
#here we provide the database encrypted password by enclosing in ENC()
#so that jasypt can detect and decrypt it
spring.datasource.password=ENC(TJ1vA+DLWFrwEmbZKmGmawEonbJw4DxhkFf53JzKfvY=)
</pre>
<br />
<br />
<h3>
Integrating Spring Boot and Jasypt</h3>
<br />
In order to instruct Spring Boot to transparently interpret our property file and extract and decrypt the encrypted properties we need to:<br />
<br />
1. Create a <code>PropertySourceLoader</code> implementation that knows how to parse property files, identify encrypted properties and decrypt them before making them available to other components. Also the class knows to get the encryption password from system properties (provided at command line by <code>-Dproperty.encryption.password=my-encryption-password</code>) or as an environment variable in the operating system (<code>export PROPERTY_ENCRYPTION_PASSWORD="my-encryption-password"</code>). Listing follows:<br />
<pre class="prettyprint lang-java">package com.myexample;
import org.jasypt.encryption.pbe.StandardPBEStringEncryptor;
import org.jasypt.spring31.properties.EncryptablePropertiesPropertySource;
import org.springframework.boot.env.PropertySourceLoader;
import org.springframework.core.PriorityOrdered;
import org.springframework.core.env.PropertySource;
import org.springframework.core.io.Resource;
import org.springframework.core.io.support.PropertiesLoaderUtils;
import java.io.IOException;
import java.util.Properties;
/**
* This class is a replacement for the default Spring PropertySourceLoader. It has the capability of detecting
* and decrypting encrypted properties via Jasypt Encryption Library.
* The decryption password must be provided via an environment variable or via a System property. The name of the property can be {@code PROPERTY_ENCRYPTION_PASSWORD} or {@code property.encryption.password}.
* For more information see <a href="https://www.blogger.com/blogger.g?blogID=2597673263725594579">http://www.jasypt.org/</a> and <a href="https://www.blogger.com/blogger.g?blogID=2597673263725594579">http://www.jasypt.org/spring31.html</a>
* For Spring Boot integration the default {@link PropertySourceLoader} configuration was overriden by
* META-INF/spring.factories file.
*
* @see org.springframework.boot.env.PropertySourceLoader
*/
public class EncryptedPropertySourceLoader implements PropertySourceLoader, PriorityOrdered {
private static final String ENCRYPTION_PASSWORD_ENVIRONMENT_VAR_NAME_UNDERSCORE = "PROPERTY_ENCRYPTION_PASSWORD";
private static final String ENCRYPTION_PASSWORD_ENVIRONMENT_VAR_NAME_DOT = "property.encryption.password";
private static final String ENCRYPTION_PASSWORD_NOT_SET = "ENCRYPTION_PASSWORD_NOT_SET";
private final StandardPBEStringEncryptor encryptor = new StandardPBEStringEncryptor();
public EncryptedPropertySourceLoader() {
this.encryptor.setPassword(getPasswordFromEnvAndSystemProperties());
}
private String getPasswordFromEnvAndSystemProperties() {
String password = System.getenv(ENCRYPTION_PASSWORD_ENVIRONMENT_VAR_NAME_UNDERSCORE);
if (password == null) {
password = System.getenv(ENCRYPTION_PASSWORD_ENVIRONMENT_VAR_NAME_DOT);
if (password == null) {
password = System.getProperty(ENCRYPTION_PASSWORD_ENVIRONMENT_VAR_NAME_UNDERSCORE);
if (password == null) {
password = System.getProperty(ENCRYPTION_PASSWORD_ENVIRONMENT_VAR_NAME_DOT);
if (password == null) {
password = ENCRYPTION_PASSWORD_NOT_SET;
}
}
}
}
return password;
}
@Override
public String[] getFileExtensions() {
return new String[]{"properties"};
}
@Override
public PropertySource load(final String name, final Resource resource, final String profile) throws
IOException {
if (profile == null) {
//load the properties
final Properties props = PropertiesLoaderUtils.loadProperties(resource);
if (!props.isEmpty()) {
//create the encryptable properties property source
return new EncryptablePropertiesPropertySource(name, props, this.encryptor);
}
}
return null;
}
@Override
public int getOrder() {
return HIGHEST_PRECEDENCE;
}
}
</pre>
<br />
2. Create a <code>com/myexample/META_INF/spring.factories</code> file to override the default <code>PropertyResurceLoader (org.springframework.boot.env.PropertiesPropertySourceLoader)</code> which is provided with the Spring Boot distribution in META-INF/spring.factories. Our file should contain one line as follows:
<br />
<pre class="prettyprint lang-java">org.springframework.boot.env.PropertySourceLoader=com.myexample.EncryptedPropertySourceLoader</pre>
<br />
That's it! Now your application should be able to use encrypted properties.<br />
<br />
Thanks for reading!<br />
<div>
Dikran<br />
<br />
<i>To give the right credits, info that helped me solving the problem and writing this post were gathered from <a href="http://stackoverflow.com/questions/24451110/creating-a-custom-jasypt-propertysource-in-springboot" rel="nofollow" target="_blank">this Stackoverflow post</a>.</i></div>
<div>
<br /></div>
<div>
<br /></div>
Dikran Seropianhttp://www.blogger.com/profile/18440451836835726018noreply@blogger.com0tag:blogger.com,1999:blog-2597673263725594579.post-66817398126238527212012-10-05T10:33:00.001+03:002019-01-14T14:35:03.959+02:00Scrum and Story Points, what's the story?After working with scrum for a while and watching this debate of time vs story points I came to a personal conclusion that helped me to make better estimations and use the story points at their best value.<br />
<br />
In my opinion story points best measure risk. You estimate this risk taking into account your proficiency, overall experience and the expertise in the technology, the project and it's business, the dependencies involved that you need to rely on to move forward (could be external systems/teams, business analysts, other people availability) and your average capacity of solving problems in a given time.<br />
<br />
So when it comes to estimating a user story you should ask yourself:
“what is the risk of this story?”
I would categorize the risk vs story points as follows:<br />
<br />
<b>1 point</b> - Virtually no risk, insignificant work doable in a very short time<br />
<br />
<b>3 points</b> - Extremely low risk, know all about it can do it quickly, probably a matter of 1-2 hours<br />
<br />
<b>5 points</b> - Low risk, know most about what I need to do, probably can fit in few hours to one working day<br />
<br />
<b>8 points</b> - Medium risk, know quite well about what I need to do, I might have some unexpected obstacles and maybe some dependencies on other (external) resources, but I am confident I can do it in 1-3 days.<br />
<br />
<b>13 points</b> - High risk, there are aspects about I have no idea on how to tackle, have external dependencies about that I am worried, it might take half of a sprint to get it done.<br />
<br />
<b>21 points</b> - Highest risk, I have right now no knowledge about the subject and no idea of how to do it, there are lots of dependencies on other (external) resources that I cannot manage, I am not sure I can solve it in a sprint so that story becomes demoable. Then you should ask yourself: is this really a good story or rather an epic? Shouldn't it better go into a research spike first so that we gather more knowledge about how to do it?<br />
<br />
However, when you estimate always take into consideration collateral aspects such as unit/functional/integration test writing (that can take as much or more than time need to code functionality), team communication, code reviews and other things that should be part of your development process.<br />
<br />
What do you think?<br />
<br />
Cheers,<br />
Dikran.Dikran Seropianhttp://www.blogger.com/profile/18440451836835726018noreply@blogger.com0tag:blogger.com,1999:blog-2597673263725594579.post-5083028179551190712012-09-25T11:10:00.004+03:002012-10-05T11:00:37.949+03:00Always Elevated Privileges in Windows 7As a developer I usually need to execute lots of commands and programs that require administrative privileges. Although my Windows user is in the Administrators group I always needed to do "Run as administrator" on command prompt, text editors and other applications that required privilege elevation, even if they were created by me or by programs that I launched!<br />
After digging a little through Windows permissions system system I found out that in order to improve security and minimize virus propagation risks, the windows team has decided that even if you are in the administrator group, and even if you are the Administrator, there is better to explicitly grant yourself the rights to do elevated privileges in an interactive way, so that underground malicious programs would not be able to go through without you knowing this.<br />
That said, I am quite sure that an experienced user can do fine even without this kind of assistance, especially in corporate environments where almost everything is filtered and secured.<br />
<br />
So, to grant yourself elevated privileges without prompt you need to:<br />
1. open security policy configuration, by typing in the command prompt:<br />
<br />
<b>%windir%\system32\secpol.msc /s</b>
<br />
<br />
A window titled "Local Security Policy" should open as below:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-jGru9PgwICw/UGFhF_M6nwI/AAAAAAAADjc/jNTVmPHxEG0/s1600/Local+Security+Policy.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="456" src="http://4.bp.blogspot.com/-jGru9PgwICw/UGFhF_M6nwI/AAAAAAAADjc/jNTVmPHxEG0/s640/Local+Security+Policy.png" width="640" /></a></div>
<br />
2. Navigate to the Security Options node:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-vJNQhhcDEHw/UGFhpFW1c2I/AAAAAAAADjk/RtUCeGq0I7w/s1600/Security+Options.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="456" src="http://3.bp.blogspot.com/-vJNQhhcDEHw/UGFhpFW1c2I/AAAAAAAADjk/RtUCeGq0I7w/s640/Security+Options.png" width="640" /></a></div>
<br />
3. On the right side click on the "Policy" table header at the top to order alphabetically the entries so that you shall have all entries starting with "User Account Control" easy to spot.<br />
<br />
4. Select the entry called "User Account Control: Run all administrators in Admin Approval Mode":<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-OtocElHUoWw/UGFkydYMynI/AAAAAAAADj8/kqHsIFk-X-U/s1600/Local+Security+Policy2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="456" src="http://2.bp.blogspot.com/-OtocElHUoWw/UGFkydYMynI/AAAAAAAADj8/kqHsIFk-X-U/s640/Local+Security+Policy2.png" width="640" /></a></div>
<br />
5. Here it is all. When activated, this option instructs Windows to ask for your permission every time when elevated privileges are required, even if you already own them. Double click on the entry and set this option to Disabled. It will require you to restart Windows in order to get effective:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-g-McuU1KVL8/UGFmz6O-EHI/AAAAAAAADkE/hZJjm8hKMlw/s1600/Local+Security+Policy3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="http://4.bp.blogspot.com/-g-McuU1KVL8/UGFmz6O-EHI/AAAAAAAADkE/hZJjm8hKMlw/s400/Local+Security+Policy3.png" width="335" /></a></div>
<br />
As a bottom line, be sure that you know what you are doing, as this will downgrade your system's overall security.<br />
<br />
Good luck!Dikran Seropianhttp://www.blogger.com/profile/18440451836835726018noreply@blogger.com2tag:blogger.com,1999:blog-2597673263725594579.post-78283294717641847532011-05-04T14:29:00.005+03:002013-07-02T09:36:19.138+03:00Tailing log files over SSH in WindowsMy problem: I wanted to be able to track logs on Unix machines using BareTail or something similar. <br />
Until now the only solution was to use putty and log console output into a local file, then open it with BareTail. <br />
Disadvantages: for each log file had to open another putty. <br />
<br />
Solution: <br />
<a href="http://dokan-dev.net/en/">Dokan SSHFS</a> (SSH File System) <br />
It's a file system mapping application that allows a remote file system map to a Windows local drive. <br />
After I installed the one I could open all 6 log files with baretail as if it was on a <br />
local drive (N). <br />
<br />
Installation Steps: <br />
1. Install the <a href="http://www.microsoft.com/downloads/en/details.aspx?FamilyID=0856eacb-4362-4b0d-8edd-aab15c5e04f5">NET 2.0 Runtime</a>;<br />
2. Install the <a href="http://www.microsoft.com/downloads/en/details.aspx?FamilyID=32bc1bee-a3f9-4c13-9c99-220b62a191ee&displaylang=en">Visual C++ 2005 Runtime</a>;<br />
3. Install the <a href="http://dokan-dev.net/wp-content/uploads/DokanInstall_0.6.0.exe">Dokan library 0.6</a> <br />
4. Install the <a href="http://dokan-dev.net/wp-content/uploads/dokan-sshfs-0201226.zip">Dokan sshfs 0.2</a>; <br />
5. Download the <a href="http://dokan-dev.net/wp-content/uploads/dokan-sshfs-0.6.0.zip">Dohan sshfs 0.6</a>; <br />
6. From 0.6 zip copy DokanSSHFS.exe and DokanNet.dll over the files installed by the 0.2 installer. This is because there is no yet an installer for the 0.6 version;<br />
7. Run DokanSSHFS.exe then choose the remote path, username and password, and choose the letter for windows drive to be assigned; <br />
8. In the options tab to check the cache disable; <br />
9. Click on CONNECT <br />
<br />
From this moment you have a new drive in Windows where you can work normally with explorer, BareTail, etc. plus an icon in the Windows taskbar that allows mount/unmount on-the-fly <br />
I hope you will find this useful. <br />
<br />
<i>Update</i>:<br />
There is an excellent log viewer called <a href="http://www.log-expert.de/" target="_blank">LogExpert</a> that can directly use tailing over sftp on *nix servers. In my opinion this is the most complete log viewer at least for Windows: highliting, regular expression filtering in a separate panel, columnizers and a lot of other useful stuff. Try it with confidence.<br />
<br />
Good luck! <br />
<br />
DikranDikran Seropianhttp://www.blogger.com/profile/18440451836835726018noreply@blogger.com2tag:blogger.com,1999:blog-2597673263725594579.post-8302266992453903022009-10-10T18:27:00.017+03:002015-05-28T12:16:20.278+03:00How to know when Java Virtual Machine is shutting down<script src="https://cdn.rawgit.com/google/code-prettify/master/loader/run_prettify.js"></script>
The question is: why would someone need to know such thing?<br />
<br />
You already have the jdk's <code>Runtime.getRuntime().addShutdownHook()</code> method that adds your thread to be called when the jvm is shutting down. So why another way?<br />
<br />
Well, I would have asked the same thing if I didn't experience an unusual situation.<br />
(For the inpatient reader that wants to skip the story, you can jump directly to the <a href="http://www.seropian.eu/2009/10/how-to-know-when-java-virtual-machine.html#answer">answer</a>)<br />
<br />
I was working on a complex web aplication that was running in a jBoss/Tomcat container. We needed a clean server shutdown in order to release the resources (connection pools, sockets, temporary files, etc). At a certain moment the team members noticed that the undeploy operation worked well while the majority of the shutdowns (but not all) were hanging at a certain point. The displayed reason was a strange exception raised from the AWT thread:<br />
<br />
<pre class="prettyprint lang-java">
Exception in thread "AWT-Windows" java.lang.IllegalStateException: Shutdown in progress
at java.lang.Shutdown.add(Shutdown.java:81)
at java.lang.Runtime.addShutdownHook(Runtime.java:190)
</pre>
<br />
We could not understand why the exception would come from the AWT as we were not using at all the AWT in our application... Or at least we didn't know it... However, who was calling the <code>Runtime.addShutdownHook()</code> and why was it driving to a jvm crash?<br />
<br />
After digging around the problem I discovered that, indeed, we were using AWT, although indirectly, by using a reporting engine. And that reporting engine was using classes of AWT to do it's job. The reporting library that we were using was packaged as a Struts plugin. The plugin's contract is simple: Struts calls the <code>init()</code> method at startup and the <code>destroy()</code>method at shutdown, on all the registered plugins. The plugin's control class was a singleton that was simply redirecting the <code>destroy()</code> calls to it's internal methods. Of course, <i>after creating an instance of itself</i>.<br />
So what? you may say. This is the normal way to do such things.<br />
Yes, only that, in this particular case, unless you used the reporting feature while working with the application, the reporting engine was not initialized until shutdown. So, at server's shutdown, Struts was calling destroy on all it's plugins and at his turn the reporting plugin, inside it's <code>destroy()</code> method, was calling <code>ReportsInitializer.getInstance().shutdown()</code>. And... Boom! JVM freeze.<br />
<br />
Ok, but why? It's just a class instance that does something within itself! What might drive it to crash all the system?<br />
Well, the nice part is just coming. So, we have a system shutdown and a Struts plugin called that is instantiating some class that uses some AWT elements inside. It does not look like something to be scared of... Only that there is a catch: when a class AWT is instantiated, the AWT Toolkit itself adds a <code>ShutDownHook</code> to the <code>Runtime</code>. I haven't yet dug inside to understand why. But, as java specs state, <a href="http://java.sun.com/javase/6/docs/api/java/lang/Runtime.html#addShutdownHook(java.lang.Thread)" target="_blank"><i>it is illegal to add a <code>ShutdownHook</code>  if the JVM shutdown sequence has already started</i></a>.<br />
Some people consider that this behavior inside AWT is a bug, because AWT should check itself if the JVM is shutting down before attempting to add its own <code>ShutdownHook</code>.
<br />
Ok. now, we found the reason why all happened. What next? I evaluated three possible solutions:<br />
- to add a generic thread exception handler and simply "swallow" this exception.<br />
- to add a myself a <code>ShutdownHook </code>and set some flag on the reporting reporting plugin not to instantiate anymore the reporting initializer.<br />
- to detect (inside the destroy method in the plugin) if the virtual machine is shutting down and skip calling the reporting initializer.<br />
<br />
The solution I chose was the last, because the generic thread exception handler couldn't prevent the exception to happen, and for the second option, the order in which JVM calls the <code>ShutdownHooks</code> is not guaranteed, so you'll never know if it will be called before or after the Struts's own <code>ShutdownHook</code>, making of the solution a non deterministic one.<br />
<br />
So I made some research to see if I may possibly know at any time if the Java Virtual Machine is shutting down. I discovered that when <code>java.lang.Runtime.exit(int status)</code> is called, it forwards the call to a class named <code>java.lang.Shutdown.exit(int status)</code>, that calls all the <code>ShutdownHooks</code> before calling the native <code>halt() </code>method. Inside this class there are fields describing the state of the system. Unfortunately the class is private package and there is no public method that returns the System's state. But luckily, in Java we have the blessed reflection. So, here comes the answer:<br />
<a href="http://www.seropian.eu/2009/10/how-to-know-when-java-virtual-machine.html" name="answer"></a><br />
<br />
<pre class="prettyprint lang-java">private boolean isSystemShuttingDown() {
try {
Field running = Class.forName("java.lang.Shutdown").getDeclaredField("RUNNING");
Field state = Class.forName("java.lang.Shutdown").getDeclaredField("state");
running.setAccessible(true);
state.setAccessible(true);
return state.getInt(null) > running.getInt(null);
}
catch (Exception ex) {
ex.printStackTrace();
return false;
}
}
</pre>
<br />
I would not advise anyone to use this on a daily basis unless he is exceptionally needing such a solution, primarily because:<br />
- the <code>java.lang.Shutdown</code> class is a package private class in the JDK and it may be changed or removed in any upcoming release. Of course, for custom built projects, that usually run for a long time on a single jdk version this is not such a big issue.<br />
- In public distributions there is a chance that the <code>SecurityManager </code>is set to forbid the access to jdk internals. But again, inside custom projects you can configure your own <code>SecurityManager</code>.<br />
<br />
I hope that you enjoyed reading this post and I am waiting for your comments.<br />
<br />
Greetings,<br />
Dikran<br />
<br />
P.S. Thanks to Alex Gorbatchev for his excellent <a href="http://alexgorbatchev.com/wiki/SyntaxHighlighter" target="_blank">SyntaxHighlighter</a>.Dikran Seropianhttp://www.blogger.com/profile/18440451836835726018noreply@blogger.com1