How to add two double with Content Automation

Had an interesting question to day from Raymond asking how to cast variable in Studio. He wants to add two double fields and put the result in another field using Content Automation. While writing your operation parameter, you might not know it but you are actually using the MVEL expression language.

MVEL is a powerful expression language for Java-based applications. It provides a plethora of features and is suited for everything from the smallest property binding and extraction, to full blown scripts. For more information, jump right to the Getting Started Guide.

Again, one of the difficulties while using Nuxeo Studio is to know which EL you can use and when. Bertrand recently wrote a very good explanation for this topic. You have to read this, seriously.

Back to the question now, Raymond was doing the following:

@{Document["calcul1:field1"]} + @{Document["calcul1:field2"]}

This will be evaluated as a String, resulting in something like “1.0+2.0″. While what we wanted was 3.0. To have a double you need to write this:

@{Document["calcul1:field1"] + Document["calcul1:field2"]}

You can do some fairly advanced things using MVEL like handling List, Arrays, use conditional expressions etc.. It’s a very powerful tool!

The post [Q&A Friday] How to add two doubles with Content Automation appeared first on Nuxeo Blogs.

The post [Nuxeo Tech Report] News from the Developer Front #8 appeared first on Nuxeo Blogs.

I’m happy to announce that Nuxeo will be doing a bug day on the 15th of every month. For the perfectionists, when the 15th is a Saturday/Sunday, the bug day will be on the following Monday :)

All of the bugs we’ll be working on are tagged with ‘bugday‘ in our Jira. The tickets that we want to include in the bug day backlog must be:
- Bugs (obviously)
- Quite short to fix (3 hours estimate max)

You’re all welcome to participate :) Send us your pull requests! And if someone external to Nuxeo fixes more bugs than any of our developer does, I’ll send him a Nuxeo t-shirt :D

If you’re experiencing difficulties while fixing bugs, you can ping us on our IRC channel (#nuxeo on Freenode). You can also join us on our Google+ community website :)…

The post Book the 15th of Every Month for a Bug Day appeared first on Nuxeo Blogs.

Document Capture is often brought up by our customers and partners as it is something that fits particularly well with Document Management. There are many solutions available on the market, but today we’ll talk about our partner Ephesoft. You can actually bridge Nuxeo and Ephesoft using CMIS. So here’s a tutorial explaining how to do that. It’s been written by Vincent Eclancher, a computer engineering school intern from AKKA technologies. Here’s what he has to say.

Digitizing a document and automatically updating it to Nuxeo with pre-filled metadata is real. To do that, we use an OCR software in which we can specify which metadata we want to extract, then connect to Nuxeo through CMIS and upload the document as a PDF with its metadata. Akka has chosen Ephesoft which has a free OpenSource version and an Enterprise version providing more functionalities. Today we’ll be using the enterprise version. Tests we did with the community version were not as good. The Enterprise version OCR engine is much better. Go to the end of this post to learn more about Ephesoft Enterprise version.

In this tutorial we are using the default Nuxeo doc types but you can of course use any custom type defined from Nuxeo Studio.
To install Ephesoft, easy and classic installation process, install the msi file. No specific configuration is required during the installation. Then connect to the Ephesoft administration interface and create a new batch by copying an existing one.

Click on Copy

Fill the different properties of the form and save it.

Batch Properties

Once the batch is created, open it by double clicking it.
Then create a document type that correspond to those you have by clicking on « Document Type » and « Add ».

Document Types tab and Add button

Then create the metadata fields. To do that, double click on the appropriate « Document Type » you just created and click on « Add ».

Metadata list of our document

Using a TIFF image uploaded in Ephesoft, we choose a KeyPattern (Date de la fiche) that will be used to detect the text to extract. Then we define where is the text to extract, the value of our metadata. We also to define a value pattern(a date regex) that will be used to validate the extracted text.

Key and Value Pattern Setup

Once the two document types have been created, we need to configure Ephesoft’s CMIS plugin. To do that, click on the « Back » button until you’re back to the main page of the batch. Then click on « Module ».
WARNING : NEVER CLICK ON THE « CANCEL » BUTTON TO GO TO THE PREVIOUS SCREEN BECAUSE IT WILL QUIT THE BATCH CONFIGURATION WITHOUT SAVING ANY CHANGES AND GO BACK TO EPHESOFT HOME.

Back button, UpRight corner of the table

Module Tab

Then double- click on the « Export » module at the end of the batch modules.

Export Module

Then on « CMIS_EXPORT » , « Edit » button.

Ephesoft CMIS Export Plugin

Give the name (and the path if necessary) of the folder where the digitalized documents will be uploaded, the address of the CMIS server, the ID of the CMIS repository and the login/password of the user doing the import.

Ephesoft CMIS Export Plugin Configuration

We’re mostly there:) We have to do the mapping between Ephesoft metadata and the Nuxeo document metadata. Edit the mapping file(DLF-Attribute-mapping.properties)located in {EphesoftPath}/SharedFolders/{batch}/cmis-plugin-mapping. First thing to do is have a link between the Ephesoft document type and the Nuxeo one, then between their metadata name :

DLF-Attribute-mapping.properties

That’s it, good to go!

About Ephesoft

As stated above, we are using the Enterprise Edition. We’ve asked Ian Pope, a friendly Welsh who likes rugby and who is Vice President, Sales and Marketing for Europe, Middle East & Africa, how to test the Enterprise version. Here’s what he sent us:

Click here to download Enterprise Edition.

About Licensing:

Once you install it, you need to email us at enterprise.support@ephesoft.com & robert.marshall@ephesoft.com and attach your license file (details.properties) located in “C:\Ephesoft\Dependencies\licensing” folder.

The install comes with pre configured batches and sample images for testing.

About Vincent
Intern at AKKA TECHNOLOGIES, 4th year at engineering school Ingésup near Bordeaux. He likes working with JAVA/JEE, Android and Web technologies. He also develops Android, iPhone and Windows Phone apps, as well as websites in his spare time.

The post Capture Documents Directly in Nuxeo Using Ephesoft and CMIS appeared first on Nuxeo Blogs.

Last week Thibaud and Alain asked me if there was an operation available to search some users and retrieve their username only, and not the complete principal object or DocumentModel. And unfortunately there is not. You can get the entries of the user directory but the result is a JSON file. This does not work for them as they wanted to use this operation to find WorkFlow assignee. So, I started writing an operation for that.

One thing you have to know about users in Nuxeo is that they are stored in a directory. So, if I do an operation to search for users, I might as well do a more generic operation that searches through any directory. This is what we’re going to do today.

To do this search, we can use the Session.getProjection method. The Session object is a directory session. It holds a connection to the database, allowing us to access, update, add or remove entries of the directory. The getProjection method is particularly interesting as it returns only one column of the table. This is exactly what Alain and Thibaud want, the username column. Once we have the result, we need to put it in the operation context to make it usable from any other operation of a chain. So my operation will have three required parameters: the directoryName, the columnName and the variableName.

As we don’t want all the entries of a directory, we need to add a filter parameter. When you’re doing a search in a directory, you can add a filter represented by a Map<String, Serializable> Java object. The key of this map will be the name of the column you are filtering, and the value will be your search parameter. You can filter any column you want. To represent key/value pairs in Studio, simply declare your param variable as Properties like this:

    @Param(name = "filters", required = false)
    protected Properties filterProperties;

The good thing about those filters is that you can specify whether they will be exact or fulltext filter (exact by default). So, we need another operation parameter to specify which field will use a fulltext match instead of the default exact match. This will be a list of String. To represent a list of String in Nuxeo Studio, simply declare your param variable as a StringList like this:

    @Param(name = "fulltextFields", required = false)
    protected StringList fulltextFields;

The operation will look like this once loaded in Nuxeo Studio:

The rest is pretty much self explanatory. Here’s the resulting operation:

/*
 * (C) Copyright 2006-2013 Nuxeo SAS (http://nuxeo.com/) and contributors.
 *
 * All rights reserved. This program and the accompanying materials
 * are made available under the terms of the GNU Lesser General Public License
 * (LGPL) version 2.1 which accompanies this distribution, and is available at
 * http://www.gnu.org/licenses/lgpl.html
 *
 * This library is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
 * Lesser General Public License for more details.
 *
 * Contributors:
 *     ldoguin
 *
 */
package org.nuxeo.ecm.automation.core.operations.services.directory;

import java.io.Serializable;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.nuxeo.ecm.automation.OperationContext;
import org.nuxeo.ecm.automation.core.Constants;
import org.nuxeo.ecm.automation.core.annotations.Context;
import org.nuxeo.ecm.automation.core.annotations.Operation;
import org.nuxeo.ecm.automation.core.annotations.OperationMethod;
import org.nuxeo.ecm.automation.core.annotations.Param;
import org.nuxeo.ecm.automation.core.util.Properties;
import org.nuxeo.ecm.automation.core.util.StringList;
import org.nuxeo.ecm.core.api.ClientException;
import org.nuxeo.ecm.directory.Directory;
import org.nuxeo.ecm.directory.Session;
import org.nuxeo.ecm.directory.api.DirectoryService;

/**
 * @author Laurent Doguin (ldoguin@nuxeo.com)
 * @since 5.7.2
 */
@Operation(id = DirectoryProjection.ID, category = Constants.CAT_SERVICES, label = "Get a Directory Projection", since = "5.7.2", description = "Executes a query using given filter and return only the column *<b>columnName</b>*. The result is assigned to the context variable *<b>variableName</b>*. The filters are specified as <i>key=value</i> pairs separated by a new line. The key used for a filter is the column name of the directory. To specify multi-line values you can use a \\ character followed by a new line. <p>Example:<pre>firstName=John<br>lastName=doe</pre>By default, the search filters use exact match. You can do a fulltext search on some specific columns using the fulltextFields. it's specified as comma separated columnName, for instance : <p>Example:<pre>firstName,lastName</pre>")
public class DirectoryProjection {

    public static final String ID = "Directory.Projection";

    private static final Log log = LogFactory.getLog(DirectoryProjection.class);

    @Context
    protected OperationContext ctx;

    @Context
    protected DirectoryService directoryService;

    @Param(name = "directoryName", required = true)
    protected String directoryName;

    @Param(name = "columnName", required = true)
    protected String columnName;

    @Param(name = "variableName", required = true)
    protected String variableName;

    @Param(name = "filters", required = false)
    protected Properties filterProperties;

    @Param(name = "fulltextFields", required = false)
    protected StringList fulltextFields;

    @OperationMethod
    public void run() throws Exception {
        // Open the Directory Session
        Directory directory = directoryService.getDirectory(directoryName);
        Session session = null;
        try {
            session = directory.getSession();
            Map<String, Serializable> filter = new HashMap<String, Serializable>();
            Set<String> fulltext = new HashSet<String>();
            // Add filters if any
            if (filterProperties != null) {
                filter.putAll(filterProperties);
                // Specify fulltext filter if any
                if (fulltextFields != null) {
                    fulltext.addAll(fulltextFields);
                }
            }
            // Do the actual search
            List<String> uids = session.getProjection(filter, fulltext,
                    columnName);
            // Put the result in the operation context
            ctx.put(variableName, uids);
        } finally {
            try {
                // Never forget to close the session you have open
                if (session != null) {
                    session.close();
                }
            } catch (ClientException ce) {
                log.error("Could not close directory session", ce);
            }
        }
    }

}

The post Write an Operation to Search through a Directory appeared first on Nuxeo Blogs.

Here’s the July tech report. Let me warn you that this one is quite dense!

• Infrastructure Improvements
• Optimizations and Misc Issues
• Automation
• UI Framework
• Products
• CI and Build

Infrastructure Improvements

Nuxeo Runtime

IDE and Deployment Model

The plan NXROADMAP-129 was to implement a white list/black list system. However, it does not seem to be needed anymore: we have found other ways to achieve the same goal.

This work includes:

• IDE bundles always override server bundles
• Make deployer deploy bundles defined as user lib in the IDE
• Make user libs/bundles part of the static class loader
• Will fix the initial deployment

Stephane will update NXROADMAP-129 accordingly. As a side note, for now we still have issues with the reload of some resources like JSF resources.

Even if we know there are solutions, it is not worth spending too much time on this: if a server restart fixes the issue, this is something that can be done when adding a new navigation case for example.

deployment-fragments

The cleanup for deployment fragment is part of the Tomcat 7 work: NXROADMAP-37.

This issue will be addressed after the Tomcat 7 alignment, hopefully in September.

Optional Contribution/Require

For dependencies, we would sometimes need to make them optional. Typically, nuxeo-poll depends on preview (which is in DM) to disable the preview, but it should work on CAP (with the needed nuxeo jars,but without the preview).

To solve this, we would need to make some requirements optional so that unresolved dependencies may not always trigger errors and break the bundle deployment.

Seam/JSF Improvements

RichFaces

Guillaume rebuilt a clean version of RichFaces including:

• Our patches
• All official patches we found

Seam/View States

Guillaume worked on NXP-11967 and NXP-11331 and thanks to that we now have:

• An automatic Conversation creation when the user opens a new Tab/Window
  • The ‘previous’ link has been removed
• A state manager that can handle correctly using Nuxeo in multiple tabs without having problem with the “can not restore view” issue

This work has been merged and validated by tests: this is a major improvement!

More Improvements to Come

We would like to improve our Seam layer to have:

• Better error handling when conversation locking issues occur
  • At least know what are the concurrent requests
  • If we need to recreate a temporary conversation: create one that can work with Nuxeo!
• Provide Nuxeo Runtime Service injection
  • This is easy and nice
• Review existing Seam beans to make them lighter
  • Use Page rather than Conversation scopes
  • Reduce invalidation requirements
  • Reduce memory footprint

An other improvement we may want to do is to provide a simple way to integrate JS/Html Widgets with the existing Widget/Layout system. Most people find it complicated to build JSF components and prefer spending time debugging HTML/JS. We should help them in doing that if we have a clean solution.

The global approach is:

• Use a simple h:inputText to create the component in the JSF tree
• Use JSON serialization to init/reset the JS widget from the model
• Use Automation RPC to do the processing
• Use valueHolder to manage transient state in JSF tree

A first tentative was done with nuxeo-select2-integration and associated task https://jira.nuxeo.com/browse/NXP-12018

Tomcat 7

The task for aligning on Tomcat 7 is still here: NXP-10071. It was first reverted because of some broken tests, and since then some overlapping work has been done on the datasources system (to support non XA mode). Julien (probably with some help from Stephane) will try to realign and merge the branch.

The goal is to have this ready for 5.7.2!

Monitoring Integration

Since 5.7.1 we have an initial integration for Metrics and Graphite. However we still have some remaining work to do.

Cleanup and Finish the Work

There are still a lot of small tasks to be completed:

• Upgrade Metrics version NXP-11995
  • Looks like we are using the version that was not properly tagged in GitHub!
• Better expose the Geronimo pool
• Rewrap application level probes NXP-11162
• Cleanup metrics NXP-11161
• Update documentation NXP-11164

JMX Integration

The data collected by Metrics is available via JMX. However, JMX:

• Is not user friendly
• Is not easy to tunnel/map as a protocol

As a result, it is a pain to explain how to get these info. This means we need an additional layer to make it easy to collect the stats. Stephane started the work in integrate a helper that includes:

• A webserver to give access to JMS via web pages
• Protocol adapters for JMX over http/https

We should(NXP-11997):

• Deploy it by default
• Only enable it via a nuxeo.conf property
• Reuse server key for authentication

Ideally, we should have all counters/probes exposed via JAX-RS.

Global Monitoring Solution

Mathieu is working on providing a global monitoring solution. So, in addition of Metrics + Graphite + Diamond, the current solution includes:

• LogStash
  • Fetch the logs
  • Clone the Metrics flow
• Riemann
  • Manage alerts (based on rules written in closure)
• Elastic Search
  • Index the logs
• Kibana
  • Front end for Elastic Search

Optimizations and Misc Issues

Storage and Files

File Upload

The current file upload widget relies on RichFaces. This implementation has some drawbacks:

• It relies on JSF/Seam
  • It maintains access to Seam conversation: this can create lock timeouts in Seam
  • Upload is done in the context of a Tx
• Upload progress is managed server side
  • Several client/server round trips
  • Progress is wrong when nuxeo is behind a buffering reverse proxy like nginx

This is an opportunity to start a test implementation for a replacement of the fileupload action:

• Just replacing the file import popup dialog is simpler than addressing the global Widget use case

The idea is to align with the way the import is currently done via Drag&Drop:

• Use Automation Batch API for upload
• Run an Operation to do the import

However, the upload part was currently done using a tweaked version of jquery-upload plugin so that we don’t have hard dependency on JS File API. A prototype was done for 5.6 and is available here nuxeo-jqueryfileupload-integration

File Copies

We have improved the way big blobs are managed to relieve the server I/O. Florent did some optimizations to avoid local copies as far as possible: see NXP-11689

S3 Binary Manager

The S3 Binary Manager can easily become the bottleneck when dealing with a lot of file access and/or big files. We saw in recent bench that S3 can indeed become a limiting factor, especially when Network config is not optimum (Nuxeo and S3 being in separated networks). To make it scale for real we need some kind of Async S3 Binary manager:

• Manage staging on a local FS
• Send on S3 on async

The implementation is not as simple as it may seem (Tiry did a very naive implementation that does not work!), since it does require some infrastructure:

• Distributed locking system
• Persistent Job system

For this we decided to use Redis (that Ben and Florent already tested): this may end up being the first Nuxeo component with a direct dependency on Redis. See NXP-11731

Async Processing

Worker and Blocking Queue

We know we need to improve the infrastructure of the WorkManager, especially in the context of a cluster infrastructure:

• Better manage synchronization between nodes
  • Manage exclusion
• Avoid running concurrent jobs hitting the same document
  • Avoid dirty updates and deadlock
• Make jobs persistent

The work on S3 Async Binary Manager may be the opportunity to bootstrap the infrastructure for that. Once we have that, Redis may become a requirement for:

• Cluster architecture
• Scalability of jobs
• Safety of jobs

This may end up being a 5.8 feature!

Long Running Processing

The Problem

We (should) all know that long running transactions are bad:

• Because it creates more concurrency against the resources like the Database
• Because it ends up generating nasty issues like
  • Deadlock
  • Dirty updates
  • Transactions timeouts

The problem is all the more important when:

• Database sucks
  • Here MS SQL Server does a very good job at being sucky
• Processes have bad granularity
  • Typically processes that directly grow with number of users or number of documents
• Processes that uses slow WebServices
• Processes that manage big files

The typical examples are:

• Async listeners that manage big file processing
• Async listeners that call WebServices
• Big Automation Chains that include a lot of processing in a big loop

Solving the Issue

Sadly there is no magic solution. The only way to avoid deadlock is to shorten the transactions. So when shortening the process is not an option, we have to split the process in smaller transactions.

So far, we have 2 patterns:

• Read/Process/Save: split the global process in 3 steps
  • Read required data inside a Tx
  • Run the long processing on disconnected objects (outside of TX)
  • Save back result in the repository inside a TX
• Create a new TX for each iteration inside a big loop

The tricky part is that we still have a low level problem where Session can not be correctly re-associated to the TX in some cases. See NXP-11681.

Asynchronous Listeners

A new base class for async listener was started using the read/process/save pattern to provide an example for a VirusScanner.

See nuxeo-virusscaner-sample and more specifically the AbstractLongRunningListener.java

The associated Jira task is NXP-11721 and as soon as NXP-11681 we will add this new base class inside Nuxeo.

Worker

Worker could also provide a base class that handles the read/process/save pattern. We could extract a base class from it and at least we should verify that the picture conversion in 5.7 uses a similar pattern to avoid long running transactions.

See nuxeo-imaging-async and more specifically PictureViewsComputerWorker.java

Task NXP-11975 was created for that.

Automation

In the context of Automation, we have an operation that runs an operation in loop. This is a good candidate for generating long running transactions, so this is a good candidate for testing the multi-tx solution. For that we created a initial implementation for RunOperationOnListInNewTransaction, see NXP-11885.

Misc Issues Reported via Support

Metrics Bad Impact on VCS Caching

Looks like VCS Cache is slowed down by Metrics timers. Stephane and Ben are investigating this point NXP-11925

Scalability Issue on FullText

As reported in NXP-11897 full text does not seem to scale well because of some migration issues on TS_VECTOR.

VCS and Dates

Most databases, even SQL Server starting with version 2005, allow us to store the time zone with datetime. However, for now, Nuxeo does not properly store the time zone with the date we save in the Database.

This means that, thanks to hacks, everything goes relatively well as long as we have the Nuxeo nodes and the database servers inside the same time zone. Fixing this won’t be very complex, but will need some tooling to manage migration.

ACL Missing Prefixes

From the very beginning, we have forgotten to use prefix inside the ACL to differentiate users and groups. This lead us to several problems:

• Having users and groups with the same name creates issues!
• We have some services that use prefix and some that don’t

This can end up generating very strange issues:

• Tasks API expect to have prefixed users/groups for assignee
• But it can be called with simple string extracted from an ACL
• Tasks assignments are not correct!

A good fix would be:

• Upgrade ACL
• Provide a migration script
• Force to use prefix everywhere

Automation

Client Side

Pseudo TCK

We have started a new documentation page that aims at defining some kind of TCK and register all known Automation Clients and their level of compliance. For now the TCK is nothing more than a suite of simple tests described in english, JS and java.

This page is just a draft for now, but we’ll be working on this.

Vlad will externalize the tests resources that are, for now, in automation-test so that we can share a simple Nuxeo plugin that only contributes more complex schemas than what we have in default Nuxeo: this is used in the TCK.

Java Automation Client and Code Sharing

The Java Automation Client is evolving at the same time as the server side. But it becomes more and more obvious that there are some classes that should be shared by the server and the client:

• Blob classes
• Properties classes
• DocumentRef classes

We also have some “almost duplicated” classes for the marshaling code. Moreover, the fact that we now support the concept of business adapter will push users to use the same class on the client and server side.

This task will not be easy to do without breaking anything on the client side API.

NXP-12010 was created for now.

JS Automation Lib

Inside Nuxeo we have an automation.js that contains primitives for doing automation calls. This script is mainly used by the Drag&Drop and some extensions. However the code was not cleaned and not documented. To address that, a new sandbox nuxeo-automation-jsshell was created.

The goal is to prepare in one module:

• A better automation.js lib
  • Improved namespace
  • Improved api
• The tests for the TCK
• A JS based Shell that could replace the applet based shell

The current code base is a mix of:

• The original automation.js
• The uploader from the Drag and Drop script
• Some new code

The first version includes tests done using qunitjs. Thomas forked the repository in troger/nuxeo-automation-jsshell to continue the work:

• Improved namespacing and API cleanup
• Switch tests to mocha and chai

For now, we have decided to keep the callback model (rather than switching to promises). Umbrella task is NXP-11860.

The target is to have everything working with Node.js, but this will require some additional work since some important API (Blob, File, XHR) are not completely available via Node.js. We’ll also work on providing Bower and Node descriptors.

Python Automation Client

We will need to work on extracting it from the Drive client.

Dart Automation Lib

Nelson is moving forward on the Dart Automation Client. Hopefully, we will also have a playground in Dart + WebUI.

See NXP-11867

.Net Automation Client

Astone Solutions is moving forward on implementing the .Net client, and they are switching the Outlook extension to use it.

Server side

Marshaling

We have a lot of code that handles JSON marshaling for Nuxeo objects:

• Inside Automation Server
• Inside Automation Client

These classes are useful everywhere: so we will factorize the code in nuxeo-core-io.

See NXP-11949

REST

Damien has started the work on making Automation API HATEOAS. This new module will be based on WebEngine:

• Using WebAdapter model
• Leveraging the shared JSON marshaling

Business Adapters

Vlad (with the help of Stephane) added support for business adapters in the Automation API. Damien tested and did some adjustment to be able to reuse the same adapter system inside the REST bindings.

Vlad added some doc with an example in confluence.

For now, the system handles only one type of Business Adapter at a time. This means you can not have an heterogeneous list as input /output, unless you create a dedicated Business Adapter to wrap the list.

We’ll see later if we have real uses like that where having support for multi-part would help…

Operation versus Chain Alignment

Vlad (with the help of Stephane) worked on aligning the models for Operations and Chains.

The idea is to:

• Be able to call Chains or Operations the same way
• Define documentation and parameters for Chains too

The final target will be to be able to bind forms to Automation Chains (ask Alain about this!).

Still to be addeed to the Chain:

• Add input/output definition (infer from first and last operations)
• Add label + description in the XMap descriptor
• Add MVEL parsing at the right place (i.e. not in the descriptor)

Impact on existing API

• Update the Flow operations (like RunChains…): done
• Update the DnD import chains to leverage new features: to be done

Studio impacts to plan:

• Add Chain Category
• Add parameter definition in chain builder
• Add doc edit in chain builder

Automation Documentation

The default site/automation/doc built-in documentation system has been updated to include the chains.

We still need to see with Lise if we can quickly make it nicer (it should not be a challenge to make it look better):

• Make a CSS that does not look like 90′s
• Provide nicer navigation (now we have a lot of operations/chains)
• Provide better display of JSON definitions

See NXP-12015.

Exception Management

Exception management in Automation will be addressed. As a first step, we’ll focus on being able to get a proper exception call stack, we’ll see for the control flow next.

UI Framework

Action System

We’ve added a type notion and a properties notion to Actions. This work is ongoing but won’t be completed before 5.7.2.

Documentation

Action types and related properties are not documented yet, but this would be easier by using widget types to render actions of a given type.

Involved JIRA task NXDOC-188

Action Widget Types

Making actions used widget types for rendering would:

• Improve pluggability: no need to override a nuxeo default template to add a custom action type
• Ease up documentation: just need to integrate action widgets types to a layout showcase-like website
• Ease up integration in Studio: action types configuration screens would be generated from widget types configuration
• Allow defining complex filterable UI elements like the widget type displaying tabs (as it needs a notion of mode to handle the content of the tab, and this notion is already handed by widget types)

Work has already been started in a branch.

Involved JIRA tasks are: NXP-11142, NXP-9673, NXP-9384

Action Filters Evaluation Context

Action filters currently rely on a restricted context.

• Make them rely on a context provided by the action, this will make actions more generic, and reusable in other UI contexts than Seam/JSF.
• Make the current document used in action filters pluggable so that widget types displaying actions do not have to be specific to it (exp: “currentDocumentActions” widget type)

This will ease up usability. Users will be able to handle standard JSF expressions in action filters. This will also make it possible to define generic widget types.

Other features that this would unblock for the future (but not for now):

• Possibility to add action widget types in layout listing rows.
• Possibility to use actions displaying tabs outside of a current document context (exp: admin center)
• Possibility to extend the action context, to provide current selected elements for instance (useful when adding action selection widget types)

Target task is NXP-10566.

Widgets

Documentation

Widgets have been added to a notion of control, that handle mostly (for now):

• handlingLabels
• addSurroundingForm
• useAjaxForm

These controls are usually handled by default layout templates and default widget types accepting subwidgets.

NXDOC-185

Widget Type Definitions

The widget type definition has already been improved, but we need to continue to extend it so that we can handle more use cases inside in a cleaner way. The requirements come from the fact that we need to handle better:

• What field types can be bound to a widget
• If widget has a hard coded title or not
• What are the default widget properties

We also need to be able to filter/present them better in Studio and improve their management.

Validation “properties”

For DAM we have requirements for adding some validation, especially on the file attributes. See NXP-11940

However, we should not try to handle this kind of validation server side: as much as possible everything should be managed on the client side. At least on recent browser this won’t be an issue and for file uploads, doing server side validation is really a pain.

Products

Nuxeo Drive

Drive, Extensions and Profiles

We added some pluggable Java Factories so that users can have a different roots layout. For now pluggability is only java/factory level.

We should add the concept of profiles:

• So that we can activate/deactivate profiles
• So that tests can cover several layouts

Antoine will update Jira accordingly.

PyQT vs PySide

Testers have experienced stability issues on the QT bindings. After doing some digging, we found out that:

• There are known bugs on PySide, the Python QT binding we use
• The PySide project is almost dead

For testing purpose we created a branch using PyQT, the original Python QT binding. This works and this seems to fix the stability issue that testers reported.

Drive and Automation

For now the Python Automation is directly part of Nuxeo Drive. With the license switch, ideally, we would like to extract the Automation Client so that it can remain LGPL.

Upload and Download

Upload and Download of big files should now be fixed. Drive client now uses the ‘automation batch upload api’.

CI Issues

Antoine took some time to fix the build and tests

• fixed some encoding on Windows
• fixed 5.7 issues with thumbs (double events in audit)
• scalability issues on recursive adapter (may explain timeout on windows servers)

The scalability issue is probably just a side effect of the test that create 20 levels deep hierarchy.

Bench

Ben is currently doing performance tests on Drive.

Next Steps

• Add support for http proxies
• Improve scan system
• Improve Drive hosting/deployment policy
• Liveedit integration

DAM

Studio Integration

We have integrated some new widgets (player, picture view, video info). Complete configuration will require to finish the work on incremental widgets.

The integration of BoxListing in DM is also in progress.

Misc issues

ImportRoot

ImportRoot doc type was renamed AssetLibrary for consistency, but we should keep ImportRoot type for compatibility.

• The new doctype is AssetLibrary to make it consistent and understandable in Studio
• We keep backward compat on ImportRoot, but there are no subtypes for ImportRoot (for now)

After having the issue on the Intranet, we created NXP-11853. For now, we’ll keep it like that, at least for the next FastTrack, even if this means adding the same subtypes to ImportRoot as AssetLibrary.

However, in the scope of 5.8, we should be able to handle automatic migration at SQL level.

Allowed Asset Types

Inside DAM import dialog, we need to filter available types based on the container. We cannot rely on default allowedTypes since there may be DM or SC types that we don’t want to be available from DAM UI.

For now, we use a dedicated DAM allowedTypes:

• This works
• This is already integrated inside Studio

So, even if this is not very clean and this probably means we should have better flexibility on the allowedTypes definitions: we’ll keep it like that for now…

NB: real option would be to extend the configuration of allowedTypes in the TypeManager

Import and DnD

The import system has been aligned on the Automation/DnD system of DM. This is still a transitional solution since:

• We’ll need to align everything on the automation.js changes
• We want to add more features to the import/dnd (like picture previews via Canvas)

Bulk Tagging

We’ll try to add support for:

• Bulk tagging
• define tags at import time (integrated with DnD)

Modernizr and HTML5 Features

For now, we are inferring browser features from the browser user agent using some server side helper. This does work for most cases, but still in some cases:

• We need the info from client side without having to call the server
• We need very precise feature detection like check history.push support that is used in REST linking

Modernizr does provide:

• Clean html5 feature detection
• Help to manage browser compatibility issues

Imaging Improvements

We know that we need to do some cleanup in Imaging components. High level list of tasks is:

• Rework PictureBook type
  • Align on ContentView
  • Align on BoxListing
  • Rethink UI and slideshow
• Cleanup adapters
  • Adapters should hold less automatic logic
  • Logic should be forwarded to service
• Adapt the picture viewgeneration to multi-tx mode
• Fix meta-data extraction:
  • Need to fix field size overflow
  • Dependency issues between meta-data/views (error on meta => rollback!)
  • Remove mistral for real
  • Integrate Laurent’s work
• Merge Nelson Silva Pull Request NXP-11582
• Make Picture views configurable.

Social Collaboration

We know we need to improve the Social Collab feature, but we must be careful not to break everything.

DM and Social Collab integration

We want to add some of the Social Collab feature to a ‘standard workspace’:

• Activity stream
• Wall

This work could also include adding the rich profile in DM. Thomas initially started this work in a branch: at some point we should resume this work and do the merge. This should end up in creating 2 separated marketplace package:

• One that adds basic social features to a Nuxeo DM
• One that include the collaboration model of the current Social Collab module

Local Groups

Local Groups are one of the cool features that comes with Social Collab, Presales often use this inside DM. Ideally, Benjamin should package this work in an addon and submit it.

Ideas on Product Evolutions

About Gadgets

Alain would like to replace/improve the OpenSocial Gadget system. However, this is not really the time to go back to plain server side rendering for dashboard: this would be anachronistic!

What we could do is:

• Improve the JS container to also support Non OpenSocial Gadgets
• Provide a way to make Non OpenSocial Gadgets using Automation + Angular

UI/UX

JS/Html Widgets

We are likely to have more and more Html/JS Widgets. We have already started doing work on jQueryUpload and Select2. This is very likely to be continued.

CI and Build

Maven 3 upgrade

We need to start this upgrade ASAP and for this we must fix the distribution issue. Julien tested to see if there are quick fixes to align our existing nuxeo-distribution maven plugin: looks like there is none.

Before starting the work on building a brand new Maven3 plugin, we should be sure that there is no alternative good solution.

• Use gradle to mix maven dependencies with procedural commands
• Fix an ant/maven plugin that does work

If we really have no other valid option that rewrite we should:

• Create the Git Repo
• Update the ReadMe to explain what we want to do
• Start some initial code (at least some tests)
• Push on Maven3 Mailing list to see if we can have more players

The post [Nuxeo Tech Report] News from the Developer Front #9 appeared first on Nuxeo Blogs.

Some time ago, in the  Nuxeo Studio Rocks – Part 0 article, I stated, “I’ll share my experience throughout this series of articles.” Thankfully, I did not make any promise about the frequency. Now, I can make one: The frequency of each release of a “Nuxeo Rocks” episode is based on a super powerful algorithm which inherits from the WhoKnows⁽¹⁾ Design Pattern.

In today’s example, we are going to build Queues using Nuxeo Studio. And as the title states the Studio rocks: Let’s work with music around. Because the Studio lets you to build about everything you want, let’s start with Status Quo, Whatever You Want⁽²⁾.

“Queue” can have different meanings, depending on the context. For example, it is “where a line of people wait. The line of people is the queue”. That’s what Wikipedia says, among other sub-meanings. But the main idea remains. Our context is the following: We want to let the end users being able to easily display a list (a queue) of documents (Nuxeo Documents). Actually, we want to let them display a list of dynamically filtered documents, nicely organized in folders, so instead of looking among billions of documents…

…they can have nice, clean folders, listing only documents they are working on:

This is why “aka Smart Folder” was added to the title. Our queues are Smart Folders actually. A folder which does not actually contains anything by itself, but dynamically displays a list of elements (here, documents), based on some filtering criteria. We already have this kind of folders everywhere: In our eMail applications, in our OS desktops, … Now, I want them them in Nuxeo.

A Case Management System is a typical example where queues are super useful. An Insurance Claim system for example, where a claim has different states (new claim received, claim opened, fetching information, under evaluation, …), and where different users/groups work on claims depending on their states. For example, users of the “Valuation Specialist” group don’t want to handle the fetching of administrative informations, and don’t have to archive the claim: They just want to work on the claims ready for evaluation, and inside this kind, they want to work only on accidents, or only on robberies, etc.

To make the example of building queues easier, I’m going to use the default DM template of the Studio. This way, you can follow the steps described here with no hassle: Just run the Studio and use the default “File” document type. No need for specific schemas, specific documents, etc. Well, as you will see in a few moment, you will need to create one new document type to build the queues, but our queues will handle only the native File document type.

For example, you will be using the default Life Cycle and the default Worklow to test your smart folder, changing the state of documents:

Here is what we want. We want to let the user create a new “Queue” folder, and enter filtering criteria for this folder. Say we want to let the user filter the folder’s content with:

• The Lifecycle State
• And the nature of the document. I’m using this item because it already exists in the Dublincore schema and we also already have a vocabulary to help the user filling the value.

So, for example, a user will be able to create a smart folder listing all approved File reports. That is: All documents whose dc:nature field is “report”, whose lifecycle state is “APPROVED”, and whose type is “File”.

The steps to create such folder are the following. We‘ll be describing them right after this summary:

1. Create a new document type, which inherits from “Folder” with its specific schema (lifecycle_state, nature) and specific layouts
   
2. Create a Content View, whose NXQL filter handles this schema, with dynamic parameters (all the trick is here)
3. Bind this Content View to the new Document type

That’s all.

Easy, right?

Now, let’s move to the details.

Starting something new? Let’s listen to New Born (Muse).

Create a new document type in the Studio. Say we name it “Queue”. It inherits from “Folder”…

…and its schema contains two fields:

The Create/View/Edit layouts are all the same and stay quite basic here:

We’re almost done (actually, the songs of the playlist are too long. Just wait and get a Coke). Now, we create a Content View. Say we name it “QueueContentView”. It is time set the NXQL filter and the result layouts (the columns to display). The NXQL filter is clearly at the heart of all this Queue/Smart Folder feature. Let’s describe what we want as a use case:

“As an end-user I want to display a list of approved Files and report nature”

Translated to Nuxeo query, it looks quite clear: We must query every Document matching the following criteria:

• The primary type must be “File”
• The lifecycle state must be “approved”
• The “nature” field of the dublincore schema must be “report”

Here is the filter in NXQL. Because we are in a Content View, we only have the WHERE clause:

Now, we want this query to be dynamic, not hard-coded as it is here. We want to use the values defined in the Queue document, in the lifecycle_state and the nature fields. So, we use the content view parameters feature, and replace the hard-coded values with a question mark. For each question mark, we add a parameter that returns the value of theses fields for the current document, which is, in this context, our Queue document. We are in the context of an EL here and must use:
#{currentDocument.schema.field}

For example:
#{currentDocument.queue.lifecycle_state}.

We also keep the usual filtering which removes the hidden/deleted documents and the proxies. Our final settings are the following:

In the “Results” tab, we define the columns to display in this view. We’re making it easy here, just the title, the nature, the lifecycle and the last modification date.

We’re almost done. Just a few seconds. The Final Count Down (Europe)⁽³⁾ looks good for this final step.

The point is: We have a Queue, Folderish document. But it displays the default Content-View and we want it to display the one we just created. So we open the definition of Queue, go to the “Tabs” tab – which is fun to pronounce, “Content Views” sub-tab and select your new content view:

We are done. Just need to test it at runtime. So let’s update the server bound to our Studio project⁽⁴⁾, and test:

• Add some documents of type “File”
• Select a nature for each of them. Select, for the testing, basic and single-word values such as “Report” or “EMail”. We’ll see below why I’m suggesting this.
• Then for some of these documents, start the default workflow, to change the life cycle state:

Now, at a workspace level for example, click the “New” button and select “Queue”. Notice how nice is our icon here (this was set in the definition of the document type):

In the creation layout, enter a value of a lifecycle state and a value for a nature. Be careful here, the NXQL query we used is case sensitive and very strict. You must enter the exact label of a lifecycle state and the exact id of a nature. This is why I suggested you to select a simple nature. For example, the id for “Report” is “report”. It is “article” for “Article”.

So, say we want our first smart folder to list all “approved File reports”. Here is how it should be created:

Save.

Enjoy.

All is good.

All is perfect!

It’s Time to Dance (The Shoes)!

. . .(dancing). . .

. . .(dancing). . .

(I just can’t refrain my body to jump and dance on the table when I hear this one)

That said, and once you have finished dancing, I may need to freeze a bit this happiness. Because yeah, ok, we have our smart folder but do you really find it so user-friendly? Hmmm? Really? Of course it is not: We don’t want the end users to know the IDs of the labels! They don’t care about IDs. They don’t care about the exact character case of the lifecycle state. This is not friendly. No-brainer: The interface must be modified.

This will be done using a vocabulary: The vocabulary for “Nature” already exists, so it’s cool. We will create a vocabulary for the lifecycle state. Then, we change the Queue layouts so they use the vocabulary. Let’s do it with the “Nature” vocabulary. First we open the Queue’s Creation Layout, then the “nature” widget, we change it from “Text” to “Vocabulary”:

Now, click the “Select Vocabulary” button and pick-up the good one (“nature”). As it is a built-in vocabulary and it is localized, you should also set the “Localize” property to true. We save, update the server(4)(again) and ta-da!

A step further? With our implementation, both fields, lifecycle_state and nature must hold a value. But maybe the user also wants to display all the Reports, or all the approved files whatever their nature. You may have notice that if you enter only one value (for example, you let the lifecycle state empty), the query finds nothing. This is because the NXQL query uses the very strict, Mister Super Serious “=” sign:

[…] ecm:currentLifeCycleState = ? AND dc:nature = ?

We must change things here. First we replace these “=” with “LIKE”:

[…] ecm:currentLifeCycleState LIKE ? AND dc:nature LIKE ?

Then, we must detect when the user lets a value empty, and if so, replace it with the joker, the “%” sign which tells Nuxeo “All possible values for this field”. So, for each parameter, we use the empty keyword and a ternary conditional expression. Something like:

#{ (empty the_field) ? '%': the_field}

In our case, the expressions are:

#{ (empty currentDocument.queue.lifecycle_state) ? '%': currentDocument.queue.lifecycle_state}

and

#{ (empty currentDocument.queue.nature) ? '%': currentDocument.queue.nature}

Now, we really are done.

We gave smartness to our folders, and they are grateful to us for this.

Waow. This was a super long blog, wasn’t it?

(1) Don’t Google this pattern. It’s my secret. Shared with the NSA but I’m not supposed to know that.
(2) The best band ever. Brit. Never miss them on stage. My playlists will always have at least one track from Status Quo.
(3) Yes. I know, I know. But Anahide (she’s in charge of the Studio at Nuxeo) really did insist for me to do a playlist in this series. This is how I take my revenge.
(4) Associating your Nuxeo DM instance to your Nuxeo Studio project

The post Nuxeo Studio Rocks – Part 1 – Queues (aka Smart Folders) appeared first on Nuxeo Blogs.

Today we released Nuxeo Platform 5.7.2, the second Fast Track version since we launched our new platform release cycles. Take a look at the release notes for the whole story. This version is coming to you roughly six weeks after the 5.7.1.

During this time we’ve been working a lot on the Content Automation APIs. This is the major focus of improvement for this release. We now have a REST API based on content automation that manipulates documents instead of operations. This will make the integration with new UI Frameworks like AngularJS even easier. And what will make this easier still is that you can also manipulate Business Objects (specialized document adapters) through this REST API.

Workflows have been improved a lot too. Now you can embed sub-workflows. This improves user experience and is much DRYer. You can also define escalation rules to trigger a chain of operations when a condition is true. These conditions are evaluated every 5 minutes by a configurable scheduler. We also added a new exclusive property to nodes. When the workflow engine runs an exclusive node, it evaluates the transitions one by one and follows the first one evaluated to true.

And we are, of course, still working on DAM. We’ve added bulk tagging, improved the asset creation workflow, and added a single asset view. Take a look at the screenshots. We have enhanced the DAM support in Nuxeo Studio. All the default document types and schemas are now available as well as default content views.

Other improvements include:  upgrade to the latest Metrics Java library and OpenCMIS versions, support for cross-origin resource sharing (CORS) through a new extension point, scan importer improvements and much more.

You can download Nuxeo Platform 5.7.2 from our website.

The next Fast Track release, 5.7.3, is scheduled for September.

The post Nuxeo Platform 5.7.2 is out! appeared first on Nuxeo Blogs.

Nuxeo World is about much much more than free tshirts and parties. In addition to the informative presentations, the product roadmap update, and the chance to chat face-to-face with Nuxeo developers, systems integrators, technology partners, and clients, there is a full day of training workshops.

Nuxeo World Workshops

On the second day of this 2-day conference, Thursday October 10, attendees can participate in hands-on, interactive workshops led by Nuxeo product managers and experts. There is a track for beginners and an advanced track, so the training material is adapted to all skill levels.

Nuxeo World Workshops

Track 1: Nuxeo Basics

• Design your application
  • Create content models based on your business problem, with the relevant metadata, validation rules, forms, lifecycle, permissions, etc.
  • Brand your application with your own logo, colors, pictures, and other custom elements
  • Define the navigation with queues, multi-axis browsing, etc.
  • Define document templates to generate fully formatted and ready-to-use documents from your content repository, such as invoices, purchase orders, etc.
  • Enable and disable existing features and services to fit your needs

• Implement your business logic
  • Create chains of operations to model your business logic
  • Design workflows to implement your processes, with fork/merge points, escalation support, due date management, dynamic sub-workflows, etc.
  • Configure user actions (actionable buttons) and event reactors triggering chains of operations
  • Configure email notifications

Track 2: Nuxeo Advanced

• Build a next generation UI for your application with AngularJS and Nuxeo
  • In this workshop, we’ll show you how to build an application with the Nuxeo Platform on the back-end and the popular JavaScript framework AngularJS on the client side. We will use the REST API to browse the repository and trigger operations on documents.
• Monitor your Nuxeo application
  • Learn how to monitor your Nuxeo application and understand extracted metrics using Graphite. Extend the monitoring framework to gather meaningful information and learn how to exploit the data using Elasticsearch and Kibana.

Fred is sporting the Nuxeo tshirt in classic black. #OnePlatformToBuildThemAll

The Nuxeo Platform is uniquely adapted to building content applications, with its flexible architecture and the complete development environment that is designed to fully support your projects. The Nuxeo World training workshops will help give you a deeper understanding of how this environment is ideal for the development of sustainable, purpose-built applications – from design and development, to deployment, to maintenance and evolution, and throughout the application’s lifetime.

Space is limited for these workshops, so register now to be sure to have access to this opportunity.

In addition to the workshop day, the first day of the conference includes information-packed sessions on topics such as Digital Asset Management, advanced workflows, AngularJS and the Nuxeo Platform, the Nuxeo Roadmap, real-world case studies of applications built with the Nuxeo Platform, and much more. The agenda is shaping up to be wicked awesome.

Early-bird registration is available until August 31st. There will, of course, be free tshirts!

The post Nuxeo World Workshops: Hands-on Learning appeared first on Nuxeo Blogs.

History

In 2011, following a simple tweet by John Vincent, a whole new movement started in the open source world. It all began with a simple statement: “monitoring sucks”.

Indeed, at the time, the open source monitoring tool set was pretty poor.

There were some commercial solutions that were pretty good for the domain they covered (HP OpenView for SNMP, Splunk for logs,…), but in the open source world the “go to” solution was Nagios, and not the others.

Nagios is not bad (and indeed we still use it at Nuxeo), but it has its limits. It’s mainly made for polling every “x” time, and it’s pretty binary in what it reports (works/broken). We needed some new tools.

John Vincent’s tweet probably came at the right time, with the right audience. It coalesced an ever-growing group of like-minded people around the idea of “we need better tools, let’s make them”.

Another important milestone in this movement was Coda Hale’s speech about metrics that pretty much says: “Don’t just check for a few predetermined things, measure and record everything, you never know when you’ll need them and what they will tell you”.

A lot has changed since then: there is a plethora of rapidly-evolving tools that see wide adoption, there has been a paradigm shift from “poll status from the application” to “push things to the monitoring tools and let them react in realtime”.

Monitoring has evolved, and it’s still changing.

What is this monitoring thing you’ve been talking about anyway?

“Monitoring” is an often used term (well, in the IT world anyway), but it’s actually used for different things. Mostly, it’s about “does my application work?”, “will it keep working?” and “what happened that made it stop working correctly?”.

Past, present, future, they are all important in the life of an application. That is the root of Coda Hale’s talk: you need to know how your application is actually behaving in the real world, see how things evolve, and from that determine what you need to watch, and plan what you need to do to keep it working. Don’t just react, be proactive so that you never need to react.

So, how does it translate in practical terms?

Mainly, record everything that you think is relevant (and possibly some that you don’t think is relevant): metrics, logs, any information you have really.

This doesn’t just have value for your operations team, it can also have business value.

For instance, if you sell an application/service but also offer a free trial, you can analyze logs to see who uses your application, which ones are customers, which ones are trials, the number of new signups for trials, then split it all in geographical locations to see which sales department needs to make a bigger push to encourage potential customers to actually sign up with you (or which ones to give up on, but that’s not something monitoring can tell you, it gives you the information, you decide how to act on it!).

The more information you have, the better you can exploit it, it’s up to you how to analyze it, the main point here is to have the information in the first place.

Ok, so I digressed a bit there. Back to down-to-earth things!

With the information you’re keeping, you can make graphs from metrics and archive structured logs data.

That can be used for post-mortem diagnosis, or to predict trends; for instance:

• When your application was not available, the database connection pool was saturated (which gives you a hint: you need an alert on this),
• At the current rate your disk will be full in 3 months so you need to place an order for more capacity (but you have a little time to do it).

So, in the end, “monitoring” is: gathering data and acting on it.

I fell asleep at the “History” part, do you have some examples?

Yes, I do!

A warning note first: the tools we use evolve very fast, the examples below may not work for you depending on the versions you use.

Let’s see how we can put the theory in practice with a basic Nuxeo installation: a single Nuxeo instance with an Apache reverse-proxy in front and a local PostgreSQL database (this is pretty much what we have on the VMs we distribute).

Collecting

We want to get the Apache access logs and the Nuxeo server log for indexing, as well as some metrics about Nuxeo, the JVM, the database and the system itself.

For the logs, the files already exist, it’s just a matter or reading new lines as they are added. We’ll use logstash¹ for this.

Logstash is a tool that can read text data from multiple sources, manipulate that data (parsing is to get structured data), then send it somewhere else. It does this through a set of inputs, filters and outputs. There are a lot of existing ones already, and it’s rather easy to add your own if you have specific needs.

Let’s see how the configuration looks for our example:

input {
  file {
    path => "/var/log/nuxeo/server.log"
    type => "nuxeo"
    sincedb_path => "/opt/logstash/.sincedb_serverlog"
  }
  file {
    path => "/var/log/apache2/nuxeo_access.log"
    type => "apache"
    sincedb_path => "/opt/logstash/.sincedb_accesslog"
  }
}

filter {
  # Add a tag to all events to identify the instance
  mutate {
    add_tag => ["mynuxeo"]
  }
  # Group all lines until they start with a new timestamp (useful for stacktraces)
  multiline {
    type => "nuxeo"
    pattern => "^20"
    negate => true
    what => "previous"
  }
}

output {
  redis {
    tags => ["logs"]
    host => "indexer.example.com"
    data_type => "list"
    key => "logstash"
  }

}

Here, we have two simple file inputs that just keep reading the files. We give them each a type which will be used later to determine which filters and outputs will be applied to them.

In the filters we start by adding a tag to identify where all those events come from, it can be something like “intranet” or “customer X” or anything that suits you really. We could just use the hostname, but hostnames can change if you move a service, so we prefer to just add a custom tag.

The second filter is used to group lines together. Keep in mind that each line in the logs will be read individually, but for a stacktrace for instance they’re not all individual events, so we want to group them together. Note that the example above is pretty basic and will not work in all cases (and it’s not year 2100 compliant, but there should be time to adapt it until then!).

Afterwards, we just send everything to a redis² server.

“Hold on! You haven’t parsed or indexed anything!” you’re going to say.

And you’d be right!

Most of the time, you are not just going to collect data from a single host, so what we do is just gather events, add some meta data to say what they are and where they come from, then ship them off to a central indexer that will do most of the work.

Another consideration is the resources usage: when you have a lot of activity, parsing and indexing can take a lot of resources, you really don’t want to do that on the same host as your production application!

In our case, we just use redis as a pub/sub queue. If you prefer there are other options such as AMQP.

We’ll come back later to logstash to see what happens next in the life of our events (or you can scroll down now if you can’t stand the suspense), let’s stick with our instance for now.

So far, we’ve handled collection of log events, but we also want metrics.

Since Nuxeo 5.7.1, we have integrated Coda Hale’s aptly-named Metrics³ library (see http://doc.nuxeo.com/x/gBDF for more details). Normally, you would just push those metrics directly to a Graphite⁴ server, but in our case we want to manipulate them first, so we send them to logstash instead.

We activate metrics in the nuxeo.conf with the following configuration:

metrics.graphite.enabled=true
metrics.graphite.host=localhost
metrics.graphite.port=2030
metrics.graphite.period=10
metrics.log4j.enabled=true
metrics.log4j.period=10
metrics.tomcat.enabled=true
metrics.tomcat.period=10

And on the logstash side we add the following in the “input” section:

graphite {
  host => "127.0.0.1"
  port => 2030
  type => "graphite"
}

To collect system and database metrics, we use Diamond⁵, which can also push metrics in the Graphite format, we just configure it to output to the same logstash input that Nuxeo uses.

At this point, we have the Apache logs, the Nuxeo logs, the Nuxeo metrics (which include the JVM ones), the database metrics and the system metrics, all sent to logstash, which forwards them to a remote server through redis.

Indexing

So, what do we do with all those events piling up in redis?

You guessed it, logstash again! It’s time to do the actual parsing and/or manipulation of our events.

We want to send different types of events to different processors, each with their own format, so based on the event type, we do different manipulations, and then send the result to the final processors.

On the collector (indexer) side, the configuration will look like this:

input {
  redis {
    host => "127.0.0.1"
    type => "redis-input"
    # these settings should match the output of the agent
    data_type => "list"
    key => "logstash"
    # We use json_event here since the sender is a logstash agent
    format => "json_event"
  }
}

filter {
  # Graphite to Riemann
  clone {
    type => "graphite"
    clones => ["riemann"]
  }
  grok {
    type => "riemann"
    pattern => "servers\.(?<host>[\w\d]+)\.(?<service>[\S]+) (?<metric>[\S]+) %{INT:epoch}\n?"
    singles => true
  }
  mutate {
    type => "riemann"
    replace => ["@source_host", "%{host}"]
  }
  date {
    type => "riemann"
    match => [ "epoch", "UNIX" ]
  }
  # Parse server.log lines
  grok {
    type => "nuxeo"
    pattern => "(?m)%{TIMESTAMP_ISO8601:date} %{LOGLEVEL:level}(?: \[%{JAVAFILE:class}\])?(?: %{GREEDYDATA:logmessage})?"
    add_tag => ["logs"]
  }
  # Parse Apache lines
  grok {
    type => "apache"
    pattern => "%{COMBINEDAPACHELOG}"
    add_tag => ["logs"]
  }
  # Fix apache logs timestamp
  date {
    type => "apache"
    timestamp => "dd/MMM/yyyy:HH:mm:ss Z"
  }
  geoip {
    type => "apache"
    source => "clientip"
  }
}

output {
  graphite {
    type => "graphite"
    host => "127.0.0.1"
    port => 2003
    fields_are_metrics => true
  }
  elasticsearch_http {
    tags => ["logs"]
    host => "127.0.0.1"
    flush_size => 100
  }
  riemann {
    type => "riemann"
    host => "127.0.0.1"
    port => 5555
    riemann_event => [
      "service", "%{service}",
      "metric", "%{metric}"
    ]
  }

}

The first half of the filters takes the graphite metrics and first duplicates them with the new type “riemann”.

Riemann⁶, is a monitoring-oriented event processor. We use it for alerting based on some of the metrics we’re going to graph, we’ll come back to it later. The main point at this part is that we are copying the graphite metrics then format the copy into a format suitable for consumption by riemann.

The second half of the filters deals with logs.

We use regular expressions to extract fields from the different logs, hence the “structured logs” part. We can further manipulate those fields to extract some information, for instance we run the geoip filter on the client IP address from the Apache logs, this will add geolocation information fields to our apache events.

Once we’ve prodded our events to give us all the information we want formatted the way the logstash outputs want, we just send them to their final destinations:

• Graphite metrics go into Graphite (surprise!)
• Riemann events go into Riemann (…)
• Structured logs go into elasticsearch⁷

Graphite will index metrics, which you can then query to make graphs of… anything you like. You can see an example dashboard using the default included front-end on the Metrics page in our documentation (linked earlier).

As mentioned before, Riemann is a monitoring-oriented event processor. We are just starting to use it and it can be kind of complex to configure (especially if you don’t know clojure), so I’m just gonna give you a simple example of how it can be used and leave the rest as an exercise to the reader!

You will remember we are getting JVM information in the Graphite metrics, for the JVM uptime it comes in the form of “server.mynuxeo.nuxeo.jvm.uptime X”. In logstash, we transformed it for Riemann into:

- host: mynuxeo
- service: nuxeo.jvm.uptime
- metric: X

We can then configure Riemann to alert us when it stops receiving this event.

First, we define a custom mailer (this goes into the main part of your riemann.config):

(def email-state (mailer {} {:subject (fn [events] (str
        "State "
        (let [state (:state (first events))] (if (clojure.string/blank? state) "up" state))
        " on "
        (:host (first events))))
}))

Then, we tell riemann to send us a mail when the state of this event changes (this goes into the streams part of your riemann.config):

    (by :host
      (changed-state
        (where (service "nuxeo.jvm.uptime")
           (email-state "system@nuxeo.com")
           (fn [event] (info "State change" event))
        )
      )
    )

(Note that this example is probably far from perfect, as mentionned, we’re only just starting with it).

This means if your Nuxeo crashes for some reason, you’ll be alerted very fast (depending on how often you configure your metrics to be sent and the time to let you configure in Riemann). If you were using “classical” polling to check if your server was still up, you wouldn’t know until the next time the “is it up?” check happens.

In Nuxeo 5.7.2 we introduce metrics based on the database connection pool, they give a good indicator of whether your Nuxeo is saturated.

You can of course also use the system metrics to create different alerts: CPU is over 80% for one minute, disk space is getting low, …

If you try Riemann out, don’t hesitate to share your configuration rules!

Last but not least, we have the elasticsearch/Kibana⁸ duo. When you send your structured log data to elasticsearch, every field you defined is indexed (with a Lucene back-end). Elasticsearch then provides you with a powerful API to search those indexes logs. Kibana is a front-end that helps you delve into your logs using this API.

By indexing structured logs from different sources (you could add the system logs in there too), it becomes much easier to correlate things. You can find out if things that return a 500 response code in Apache all come from the same IP, or from the same country, or what requests happen at the same time as you get stacktraces in your Nuxeo logs, or pull some statistics out of your logs. As elasticsearch is very fast, you can start with a broad request and then fine-tune it on the go.

Once you are confortable with it, you can also make near-realtime dashboards out of it, here is an example of what we did while trying out Kibana v3 and the GeoIP fields:

Conclusion

What all those tools aim to do is give you easy access to your information. Information is not knowledge, but it’s the basis for it, you just have to find out how to exploit it! Give those tools a whirl (or others, there are plenty of fine ones that we didn’t mention), you won’t regret it!

^{[1] http://logstash.net/↩}

^{[2] http://redis.io/↩}

^{[3] https://github.com/codahale/metrics↩}

^{[4] http://graphite.wikidot.com/↩}

^{[5] https://github.com/BrightcoveOS/Diamond↩}

^{[6] http://riemann.io/↩}

^{[7] http://www.elasticsearch.org/↩}

^{[8] http://kibana.org/ (v2) or http://three.kibana.org/ (v3)↩}

The post Monitoring (at) Nuxeo appeared first on Nuxeo Blogs.

It’s been a while since we have written about Nuxeo Studio, so here’s a summary of the work we’ve done for versions 2.12 and 2.13.

Digital Asset Management

Major improvements have been made on our digital asset management module in the Nuxeo Platform 5.7.x versions. And many of these improvements were geared towards Studio:

• DAM-related document types are now available in Studio, so it is easier to inherit or use them in customizations with Studio. The newly available document types are: Video, Picture, PictureBook and Sound.
• DAM facets and schemas are now available: Picture, Video, Audio (https://jira.nuxeo.com/browse/NXS-1215).
• It is now possible to choose if a document should or not be available in DAM through a specific facet “Asset” (available when the DAM package is selected in Project Settings, starting with Nuxeo Platform 5.7.1) (https://jira.nuxeo.com/browse/NXS-1629).
• New DAM search filters can be created by simply selecting the DAM flag on a content view. (starting with Nuxeo Platform 5.7.1)
• New DAM-specific action categories are available to make it easier to add user actions (buttons) to the DAM interface. (starting with Nuxeo Platform 5.7.1)(https://jira.nuxeo.com/browse/NXS-1456)
• DAM-specific tabs can be filtered from Studio, e.g. the metadata tab in Pictures. (https://jira.nuxeo.com/browse/NXS-1457)
• And of course the Nuxeo DAM default Application template has been updated to reflect the changes made in 5.7.2.

DAM Types and Facets

Workflows

Nuxeo Platform 5.7.2 comes with many workflow improvements, which are reflected in Nuxeo Studio 2.13:

• New option to make exclusive nodes in a workflow. When a node is set as “exclusive”, its outgoing transitions can be ordered and only the first true condition will be followed. This prevents parallel execution in a workflow even with complex transition conditions. (starting with Nuxeo Platform 5.7.2)
• New sub-workflow node. It is now possible to embed a workflow into another one and pass it specific inputs from the parent workflow. This makes it possible to reuse generic sub-workflows into bigger ones. (starting with Nuxeo Platform 5.7.2) (https://jira.nuxeo.com/browse/NXS-1614)
• New workflow escalation rules enable an operation chain to run at regular intervals based on certain conditions. The default interval is 5 minutes. For instance, you can send an email if a task is overdue. Of course, an escalation rule can also be set to run only once. (starting with  Nuxeo Platform 5.7.2) (https://jira.nuxeo.com/browse/NXS-1619)
• New workflow operations are available, making it easier to use workflows in operation chains. Examples: Start a workflow, Cancel a workflow, Get open workflow tasks. (starting with Nuxeo Platform 5.7.2) (https://jira.nuxeo.com/browse/NXS-1615)

New Escalation Rules Tab

Miscellaneous

• Operation chains can now have inputs other than Documents, making it easier to create advanced business logic with Studio. (https://jira.nuxeo.com/browse/NXS-931)
• It now possible to configure which refresh and reset events are used by the content view, for more control over the reaction to changes.(https://jira.nuxeo.com/browse/NXS-1659)
• Content view filters can now be unfolded by default. (starting with Nuxeo Platform 5.7.2) (https://jira.nuxeo.com/browse/NXS-1663)
• Several improvements to content view configuration. (https://jira.nuxeo.com/browse/NXS-1480)

Release Notes

• Nuxeo Studio 2.12 Release Notes

• Nuxeo Studio 2.13 Release Notes

Connect Upgrade

If you’re using Nuxeo Studio, it means you’re also a Nuxeo Connect user. So you might have noticed some changes. The main new feature is a new dashboard (written with AngularJS and Content Automation) available for Beta testing. This should let us add new features for Connect users more easily. Let us know what you think — there is a new contact form for this. So don’t hesitate to use it and send us your feedback :)

The post What’s New in Nuxeo Studio 2.13 appeared first on Nuxeo Blogs.

Here’s a fun project to help you learn more about working with the Nuxeo Platform — display a random background image on the login page. And of course we’ll be picking up the image into our Nuxeo instance. Part of the challenge is to access this image when there is no user session :) This an opportunity to learn more about OpenURL and UnrestrictedSessionRunner. Here’s how we’re going to do this.

Create a WebEngine Module

What we need is to return an image for a simple URL. A WebEngine module is the simplest and quickest solution to achieve that. So using Nuxeo IDE, I’ve created a new WebEngine module called RandomLoginPage. It doesn’t do much for now. If you deploy it on your server and go to yourserver/nuxeo/RandomLoginPage/, you should be redirected to the login page. Once you login, you see a simple web page explaining hte module was created with Nuxeo IDE. Before searching for a random image, let’s remove the authentication need for this specific URL.

Nuxeo OpenURL Extension Point

The Nuxeo OpenURl extension point is used to define URLs that can be accessed without authentication. Which is exactly what we need!

URLs are declared using a pattern. You can insert properties variables like the context path declared in nuxeo.conf. You can also use regex to avoid repeating yourself like this:

    <extension
      target="org.nuxeo.ecm.platform.ui.web.auth.service.PluggableAuthenticationService"
      point="openUrl">
      <openUrl name="userReg">
      <grantPattern>/nuxeo/site/userRegistration/validate/.*</grantPattern>
      </openUrl>
    </extension>

Here all the URLs after /nuxeo/site/userRegistration/validate/ will be accessible without authentication.

This is my contribution for my WebEngine module:

    <!-- Following URLs don't need user authentication -->
    <extension
      target="org.nuxeo.ecm.platform.ui.web.auth.service.PluggableAuthenticationService"
      point="openUrl">
      <openUrl name="RandomLoginScreen">
        <grantPattern>${org.nuxeo.ecm.contextPath}/site/RandomLoginPage/</grantPattern>
      </openUrl>
    </extension>

Now you should be able to access your page yourserver/nuxeo/RandomLoginPage/ without having to login. We can start searching for an image.

Finding Picture Assets

To retrieve a document, we need a CoreSession object. It’s the link to the content repository. When using WebEngine, you have access to it from the context object. Here’s how I would do it:

package org.nuxeo.sample;

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.Response;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.nuxeo.ecm.core.api.ClientException;
import org.nuxeo.ecm.core.api.CoreSession;
import org.nuxeo.ecm.core.api.DocumentModel;
import org.nuxeo.ecm.core.api.DocumentModelList;
import org.nuxeo.ecm.core.api.blobholder.BlobHolder;
import org.nuxeo.ecm.webengine.model.WebObject;
import org.nuxeo.ecm.webengine.model.impl.ModuleRoot;

@Path("/RandomLoginPage")
// the returned object will be an image, and yes I only have PNG images :D
@Produces("image/png")
@WebObject(type="RandomLoginPage")
public class RandomLoginPage extends ModuleRoot {

    public static final Log log = LogFactory.getLog(RandomLoginPage.class);

    @GET
    public Object doGet() throws ClientException {
        // retrieve the coreSession from the WebEngine context
    	CoreSession coreSession = ctx.getCoreSession();
        // Query all the Picture documents
    	DocumentModelList pictures = coreSession.query("Select * From Picture");
        // Generate a random number
        int size = pictures.size();
        if (pictures.isEmpty()) {
            return Response.status(404).build();
        }
        int nbPicture = (int) Math.floor(Math.random() * size);
        // Retrieve the blob from the picture document with the BlobHolder Adapter
        DocumentModel pictureDoc = pictures.get(nbPicture);
        BlobHolder bh = pictureDoc.getAdapter(BlobHolder.class);
        return bh.getBlob();
   }

}

But the thing is, this won’t work :) The CoreSession object will be null because we don’t have any user logged in. And no user means no CoreSession.

Get a CoreSession when you don’t have a user

Usually when you search for documents in Nuxeo, you use the logged in user’s CoreSession. When you’re writing code in a Seam bean, in an operation or a listener, there is always an easy way to get your hands on the user’s CoreSession. Here we still have no user. So the trick is to use the UnrestrictedSessiontRunner. This class abstracts everything you need to get a CoreSession. So let’s create a new class, FindRandomLoginScreenUnrestricted, that extends the UnrestrictedSessionRunner class. You’ll have to implement the run method as follow:

package org.nuxeo.sample;

import org.nuxeo.ecm.core.api.Blob;
import org.nuxeo.ecm.core.api.ClientException;
import org.nuxeo.ecm.core.api.DocumentModel;
import org.nuxeo.ecm.core.api.DocumentModelList;
import org.nuxeo.ecm.core.api.UnrestrictedSessionRunner;
import org.nuxeo.ecm.core.api.blobholder.BlobHolder;

public class FindRandomLoginScreenUnrestricted extends
        UnrestrictedSessionRunner {

    protected FindRandomLoginScreenUnrestricted(String repositoryName) {
        super(repositoryName);
    }

    private Blob b;

    @Override
    public void run() throws ClientException {
    	DocumentModelList pictures = session.query("Select * From Picture");
        // Generate a random number
        int size = pictures.size();
        if (pictures.isEmpty()) {
            return;
        }
        int nbPicture = (int) Math.floor(Math.random() * size);
        // Retrieve the blob from the picture document with the BlobHolder Adapter
        DocumentModel pictureDoc = pictures.get(nbPicture);
        BlobHolder bh = pictureDoc.getAdapter(BlobHolder.class);
        b = bh.getBlob();
    }

    public Blob getBlob() {
        return b;
    }
}

To be instantiated, a runner needs a repository name or an existing CoreSession. Here’s how to retrieve the default repository name:

    private static String defaultRepositoryName;

    @GET
    public Object doGet() throws ClientException {
        FindRandomLoginScreenUnrestricted runner = new FindRandomLoginScreenUnrestricted(
                getDefaultRepositoryName());
        runner.runUnrestricted();
        Blob b = runner.getBlob();
        if (b == null) {
            return Response.status(404).build();
        }
        return b;
    }

    private String getDefaultRepositoryName() {
        if (defaultRepositoryName == null) {
            try {
                defaultRepositoryName = Framework.getService(
                        RepositoryManager.class).getDefaultRepository().getName();
            } catch (Exception e) {
                throw new RuntimeException(e);
            }
        }
        return defaultRepositoryName;
    }

Now if you go to yourserver/nuxeo/RandomLoginPage/, You should have your image. The next step is to display it on the login screen.

The loginScreen Extension Point

Our next goal is to modify the login screen do display our image instead of the default one. We could overwrite the login.jsp page and hard code our new URL. But we won’t do that, because that would be BAD. You need to avoid as much as possible to overwrite templates in Nuxeo. Extension points are made to help us. In our case, we need to use the loginScreen extension point. It lets you define custom styles for the login box, the footer and header, a new logo, the newsBox content, bodyBackgroundStyle style, OpenId providers and much more… Right now we just need the bodyBackgroundStyle option:

  <extension
    target="org.nuxeo.ecm.platform.ui.web.auth.service.PluggableAuthenticationService"
    point="loginScreen">
    <loginScreenConfig>
      <bodyBackgroundStyle>url("${org.nuxeo.ecm.contextPath}/site/RandomLoginPage/") repeat scroll bottom left #ffffff;</bodyBackgroundStyle>
    </loginScreenConfig>
  </extension>

Now you should see a random image taken from the content repository each time you load the login page :)

You may not know it, but this is an image chosen randomly from my Nuxeo instance :)

The post [Monday Dev Heaven] Pick a Random Picture as Background for the Login Page appeared first on Nuxeo Blogs.

Welcome to the August tech report. This month we have updates on our UI/UX Framework and Content Automation.

• UI/UX Framework
• Content Automation
• Misc Platform Issues

UI/UX Framework

New Widgets

Select2 Integration

Guilaume took over the integration of Select2 based widgets. We will have 6 widget types:

• suggestDocument: single/multi
• suggestDirectory: single/multi
  • Built-in support for nested vocabularies
• suggestUser
• suggestGroup

These widgets will be integrated as a new nuxeo-jsf module in 5.7.3. Select2 widgets will be used inside the default screens:

• Metadata layout with Nature/Subject/Coverage
• Faceted search
• Relation screen

Part of the goal of this set of Select2 widgets is also to fix the problem we have with the current suggestion and chained vocabularies widgets. Anahide and Guillaume will create the Jira issues about all problematic use cases:

• Integrate use case in Layout Demo
• Update widgets and Studio

Anyway, even if the use cases are complicated, we need as much as possible to keep the meta-model simple.

Note: It is probably better to leverage the fact that users can override JS rendering callbacks, rather that creating widgets with 23 properties.

SafeEdit

Playing with Select2, we discovered that SafeEdit is not generic enough to also handle the case of Select2. Guillaume will extend the SafeEdit JS API so Select2 widgets can register themselves.

jQueryUpload

We now have in a sandbox a simple integration of jQueryFileUpload with Automation API that can be used to replace the import file dialog. Switching from RichFaces upload to jQueryUpload provides:

• Client side progress management on recent browsers (not MSIE 7,8,9)
• Upload outside of Seam context (no more Concurrent calls to conversation)
• Skinable UI
• Drag&Drop support
• Plugins to manage file preview

The sandbox module has been realigned to 5.7.3 and the goal is now to integrate it by default:

• Replace the import file fancybox dialog
• Replace the files tab

Ideally, we’ll also want to package it as a real widget and use it to replace the file widgets.

Seam Injections

As discussed earlier, we have started building our own version of Seam: NXP-12207

For now, our branch was started from 2.1.SP1:

• Reintegrate the interceptors removal
  • Avoid the warns at startup
• Allow to directly inject Nuxeo Runtime components and Services inside Seam beans
  • Remove the “Delegates”

Having our own version of Seam will also open the way for some other changes we may want to do:

• Avoid the “Transaction Failed” message displayed by Seam when we rollback a transaction
• Better manage conversation concurrency
• Better manage context demarcation

Widgets Everywhere

Anahide is doing work on the WidgetDefinition to allow better configuration and code sharing. Evolutions on the WidgetType definition include:

• Being able to define what type of field can be bound to the widget
• Being able to define a default value

Operation Docs and WidgetDefinition

The current Operation definition can contain widgets that are used by Studio to know how to generate the form to let the user enter parameters.

Typically:

@Param(name = “language”, required = false, widget = Constants.W_OPTION, values = { NXQL.NXQL })
protected String lang = NXQL.NXQL;

For now these widgets are defined inside Automation itself, and have nothing to do with Nuxeo Layout/Widget system, this is a problem:

• For understanding: the widget names and types are not the same
• For code sharing inside Studio: Studio has to handle 2 types of widgets

This improvement as well as the addition of a description for the parameter will allow:

• Studio UI improvements
• Automation Doc website improvements

Widgets and Actions

In order to make the contribution of incremental layouts easier (less indirections Layout -> Category -> Action -> Widget), Anahide will allow to directly contribute a WidgetType inside the Action.

Action System and Context

The long awaited work to cleanup the way we build the Action Context has been done: NXP-10566

Basically:

• JEXL is out
  • Still used by runtime scripting: need to check if this is useful
• Context initialization has been standardized
• Seam context lookup is no longer a hack

Note: we still have nuxeo-platform-el, based on Unified EL, used in the audit and shiboleth source code.

Automation

Automation Evolutions

Params vs Context

With NXP-11096, Vlad started to allow passing parameters to Chains. Part of the implementation imply to allow fallback of empty parameters to Context entries with the same name.

We should go further:

• Manage that at operation level
• Review existing operation to be sure that all parameters are explicit

Feedback Messages

In a lot of cases, Automation chains need to generate user messages. Currently we have 2 options:

• Return messages in the context with a known name
  • This will be displayed in JSF if the Automation Chain is called from the JSF UI.
• Use a dedicated operation
  • Need to explicitly initialize Seam context first

And anyway:

• Looking up the translation map is a pain
• Passing parameters to the message is not always easy

We should provide a global solution for these use cases:

• Provide a simple API inside Automation context
  • Add message with parameter
  • Translate a string
  • Get pending messages
• Store the required info inside the context
• Automatically flush the messages depending on the available context
  • Inside JSF if available
  • Inside the log
  • Inside Angular?

Vlad will check that and create the Jira issue.

RESTful API

The sample API (as you saw this week) has been improved. Next steps include:

Extend contextParameters

contextParameters are currently used in DocumentList serialization to be able to add URLs and links that can be used on the client side to save remote calls. This was introduced for OpenSocial use cases. With the Rest API, we would like to ask the server to add information inside the documents we fetch:

• Get BreadCrumb info
• Get Available actions
• Get related documents
• Get tags

This means that we should have a pluggable contextParameters management and be able to register on the server side some contextParameters providers.

Batch/Bulk

Ideally, the rest API should provide a way to do bulk operation…

Android Connector

We have quickly defined the main tasks for the next weeks, based on:

• What we know should be done!
• Changes we have done in Automation on the Java side
• Feedback from Julien’s use cases

Cleanup Build

Align on a recent SDK using Google Maven repository: NXMOB-43

The work was started with Nicolas/Julien but needs to be finished.

Automation Client Alignment

The Android Automation Client is basically a fork of the original Automation client. As is, it contains features that are not in the Java client:

• Caching
• Off-line
• Sync
• Upload/Download manager

And of course, on the contrary, there are features of the new Automation Client that are not supported:

• Jackson JSON
• Business adapters

The task NXP-7804 exists for a long time to track this. Vlad will try to take a look with Nicolas, to see in more details what can be done without having to spend too much time on it.

Authentication

The Android Automation client should support:

• Token based authentication (the one created for Drive)
• OAuth2

See NXMOB-44

Review Usage of Android SDK Concepts

The current Android Connector was designed on an old version of Android SDK. We should review the current architecture to be sure we leverage the possibilities of Android SDK:

• Review how Nuxeo is exposed as a ContentProvider
• See if we should package part of the code as an IntendService
• Support for Fragments -…

Vlad will try to take a look with Nicolas.

Fix Activity LifeCycle Management

According to tests that have been done by Nicolas and Julien was have the following issues:

• Init phase is done with a wrong settings (default one)
• Shutdown can leak connections
• Orientation switch can generate crash

See: NXMOB-46

Layout Manager

We want to improve the existing Android Layout manager:

• Review widget binding system
• Leverage new android widgets
• See if we can leverage the layouts in the list views

See: NXMOB-47

Documentation

The Automation Documentation is really becoming a mess. To improve things, we should:

• Make Http API a first level entry in the documentation
• Make an overview page that presents the 3 subset of http API
  • Historical REST API: get doc, download, export…
  • Automation JSON-RPC API
  • New Automation REST/HATEOAS bindings
• Be sure we don’t have too much resources dispersion between Studio and Dev docs

Automation Bindings

Dart by Nelson Silva

Nelson sent us some very good news that I’ll share with you:

I’ve been updating my GitHub, Drone.io and pub packages for the Dart Nuxeo Automation client so you can now checkout:

• GH project: https://github.com/nelsonsilva/nuxeo-dart-automation
• Pub package: http://pub.dartlang.org/packages/nuxeo_automation
• API Docs: http://nelsonsilva.github.io/nuxeo-dart-automation/nuxeo_automation.html

I’ll work some more on these as soon as I have an initial version of the sandbox working with polymer.dart.

For now I have just uploaded the initial version to https://github.com/nelsonsilva/nuxeo-dart-automation-sandbox

I’m just using the exec-maven-plugin to package the Dart application so it should be easy to integrate with your CI server.

I’ve also set up drone.io to package the sandbox https://drone.io/github.com/nelsonsilva/nuxeo-dart-automation-sandbox

Misc Platform Issues

CI and Build

Maven 3 Upgrade

Still needs to be addressed ASAP.

Marketplace Publishing from Jenkins

This is done and this seems to work!

DAM

Local Configuration/Flavors

Currently inside DAM, if people have defined flavors on some container, when using AssetBrowser view and navigating between the assets of different containers, it can trigger a flavor change (in Ajax) that will basically break UI.

We could investigate to fix this, but basically: we don’t care about local flavors for AssetBrowser. This means the DAM view should not support placeful flavors.

Bulk Tagging

DAM bulk edit support tagging (using a Seam even to forward info). This code about bulk tagging should be added into DM. Since, Tagging is part of DM whereas Bulk Edit is part of CAP, we need a way to contribute the tagging widgets to the bulk edit layout. This is a good use case for incremental layouts!

For now Thomas is adding automatic versioning inside the BulkEdit. But we still have to figure out how we better manage automatic version creating in a consistent way.

Infrastructure

WebService/CXF

Arnaud is working on NXP-11921 to switch Nuxeo from Metro WS to CXF:

• This is good since CXF is better that Metro
• This may help replacing the Jersey JAX-RS impl

For now, there are issues with Platform Sync: this may be a good opportunity to check this code and see if we can manage to find a way to test it automatically!

The post [Nuxeo Tech Report] News from the Developer Front #10 appeared first on Nuxeo Blogs.

We recently opened a Nuxeo account on Crowdin.

Crowdin is a translation and localization management platform that handles both documents and software projects.

We’re trying to make it easier to manage existing and new translations. As of now, you only have the default English messages.properties file to translate. It contains the labels found in a regular Nuxeo Platform download. Some labels from specific add-ons might be missing. We’ll add them soon. In the meantime you can already join the Crowdin project by going to the following URL:

http://crowdin.net/project/nuxeo/invite

When you join a project, you have the translator status, which mean you can propose a new translation for any label. I am looking for volunteers to become proofreaders. Proofreaders validate the proposed translations. Capgemini Netherlands is already taking care of the Dutch translation through this system.

If you want more details about the translation process, please read the how to translate Nuxeo and the Contributing to Nuxeo pages on our documentation site.

About dialects (like French Canadian for French), don’t worry if they are not 100% fully translated. Crowdin will automatically fall back to the base language during the export.

We will take care of the Crowdin export to the nuxeo-platform-lang-ext before each release. This will be done manually at first, but should be automated in the future.

Feedback is welcome!

The post Managing Nuxeo Platform Translations with Crowdin appeared first on Nuxeo Blogs.

We recently had a very nice question from Utopian who asks how he can become a documentation contributor. I love those kind of questions. I’ve already written something about translation contribution today, you can check it out here.

So about becoming a documentation contributor, if you want to help, like Utopian, here’s how it works.

First I have to tell you that we have developer, administrator and user documentation. So everyone in the community can contribute. You don’t have to be a very technical person :-)

Now for the practical details. If you want to contribute to the documentation, you obviously need to create an account on doc.nuxeo.com. Then, make yourself known. You can do this on Nuxeo Answers, the nuxeo-dev mailing list or on the Nuxeo Google+ community page. Once we’ve seen you, we can add you to the contributors group. It means you’ll be able to edit the user, developer and administrator guides.

When you edit a page, its state will go to draft. A draft can only be seen by you, the Nuxeo group and the administrators. Our technical documentation writer will review your changes and possibly leave some comments. When the page is ready, it’s published by someone at Nuxeo. Your changes are now visible for everybody.

To sum up today’s blog posts, if you want to translate the documentation, please send us directly an email so we can find the best way and tools to do it. Thanks!

The post [Q&A Friday] How to Contribute to Nuxeo Documentation appeared first on Nuxeo Blogs.

How to Add JavaScript or CSS Resources to Your Pages

Today we have a question from jvinai who asks how he can add jQuery UI resources into his project.

To do this you need to know a little bit about the theme engine:

The theme is in charge of the global layout or structure of a page (composition of the header, footer, left menu, main area…), as well as its branding or styling using CSS. It also handles additional resources like JavaScript and CSS files.

As usual, this works with extension points. The first thing to do is declare your resource. It can be a CSS or a JavaScript file.

  <extension target="org.nuxeo.theme.styling.service" point="resources">
    <resource name="static_styles.css">
      <path>css/static_styles.css</path>
    </resource>
    <resource name="jquery.fancybox.js">
      <path>scripts/jquery/jquery.fancybox.pack.js</path>
    </resource>
  </extension>

The next step is to state where you want your resource to be loaded. You need to choose the fragment where you want to have it (a page is made of fragments). If you want it to be loaded in every page, I would suggest you put your resource in the include fragment. Notice the merge="true" attribute. If you don’t use it, you will override the default include completely.

  <extension target="org.nuxeo.theme.services.ThemeService" point="views">
    <view name="nuxeo5 includes" template-engine="jsf-facelets" merge="true">
      <format-type>widget</format-type>
      <resource>jquery.fancybox.js</resource>
      <resource>static_styles.css</resource>
    </view>
  </extension>

Make sure that your resources are copied in the war using the deployment-fragment.xml:

<?xml version="1.0"?>
<fragment version="1">
  <install>
    <!--  unzip the war template -->
    <unzip from="${bundle.fileName}" to="/" prefix="web">
      <include>web/nuxeo.war/**</include>
    </unzip>
  </install>
</fragment>

Now, you can deploy your code and test it. If you look at the source code of the page, you won’t see the resource as it is. The theme engine has minified it along with the other resources.

The post [Q&A Friday] How to Add JavaScript or CSS Resources to Your Pages appeared first on Nuxeo Blogs.

We recently received the many photos taken during Nuxeo World 2013. And I needed a way to see all the photos easily. Meaning I did not care about the metadata, I just wanted to see the crazy faces of my fellow co-workers. So I decided to do a small browsing module. Which is the perfect opportunity to try AngularJS along with our REST API.

My goal is to be able to see all the pictures having a particular tag. The pictures have to be small and have to be viewable full size when clicking on them. And since we have a lot of pictures, I’ll add infinite scrolling.

This first post will focus on how to use the REST API and the search and Business Object (BO) adapters. I’ll write about the AngularJS integration in the second one.

How to Retrieve Documents with the API

So how do I get my documents given a facet (Picture) and a tag? Nuxeo has a built in search adapter. It lets you run an NXQL query or a fulltext search. This adapter has to be applied on a document. This might not sound particularly logical. Why would you need a document when you’re actually trying to get one? The answer is because a document has an associated repository. And when you run a query, you do it against a particular repository. So it’s just a way to choose against which repository your query will be run.

The simplest way to get a document with the new API is:

http://localhost:8080/nuxeo/api/v1/path/

This retrieves the root document of the default repository using its path ‘/’. The result should look like this:

{ "changeToken" : null,
  "contextParameters" : {  },
  "entity-type" : "document",
  "facets" : [ "Folderish" ],
  "isCheckedOut" : true,
  "path" : "/",
  "repository" : "default",
  "state" : null,
  "title" : "7eaa8a0c-a0d8-4120-9aaa-c6cf80088d6f",
  "type" : "Root",
  "uid" : "7eaa8a0c-a0d8-4120-9aaa-c6cf80088d6f",
  "versionLabel" : ""
}

We have a document, now we can use the @search adapter. Using an adapter on a resource is easy. This is a resource:

http://localhost:8080/nuxeo/api/v1/path/

To use the search adapter, simply add @search to you URL:

http://localhost:8080/nuxeo/api/v1/path/@search

This adapter takes several query parameters. We can split them in two categories: those related to the search and those related to pagination.

About the Search

– query: Any valid NXQL query. If you don’t specify this parameter, Nuxeo will try to run a fulltext search using the fullText query parameter.
– fullText: The search term for your fulltext search.
– orderBy: An optional Order By clause will be append to your fulltext search.

This will run a the SELECT * FROM Document query against the default repository:

http://localhost:8080/nuxeo/api/v1/path/@search?query=SELECT * FROM Document

This will run a fulltext search for the word searchTerm. Results will be ordered by modification date.

http://localhost:8080/nuxeo/api/v1/path/@search?fullText=searchTerm&orderBy=dc:modified

The NXQL query will actually be SELECT * FROM Document WHERE ecm:fulltext = “searchterm” AND ecm:isCheckedInVersion = 0 AND ecm:path STARTSWITH “/” ORDER BY dc:modified.

If your document is the default domain instead of the root:

http://localhost:8080/nuxeo/api/v1/path/default-domain/@search?fullText=searchTerm&orderBy=dc:modified

Your query will look like SELECT * FROM Document WHERE ecm:fulltext = “searchterm” AND ecm:isCheckedInVersion = 0 AND ecm:path STARTSWITH “/default-domain” ORDER BY dc:modified.

About the Pagination

When the API returns a DocumentList, this list can be paginated. Here are the query parameters available to control the pagination:

– currentPageIndex: Defaults to 0, the index of the page you want.
– pageSize: Defaults to 50, the number of entries on a page.

Here’s an example result:

{ "currentPageIndex" : 0,
  "currentPageSize" : 20,
  "errorMessage" : null,
  "hasError" : false,
  "isLasPageAvailable" : true,
  "isNextPageAvailable" : true,
  "isPaginable" : true,
  "isPreviousPageAvailable" : false,
  "isSortable" : true,
  "maxPageSize" : 100,
  "numberOfPages" : 7,
  "pageCount" : 7,
  "pageIndex" : 0,
  "pageSize" : 20,
  "resultsCount" : 128,
  "totalSize" : 128,
  "entity-type" : "documents",
  "entries" : [ { "changeToken" : "1382970456847",
        "contextParameters" : { "documentURL" : "/nuxeo/site/api/v1/id/e9e1035f-0535-4aed-bb47-f5fa429bfc65" },
        "entity-type" : "document",
        "facets" : [ "Picture",
            "Commentable",
            "Asset",
            "MultiviewPicture",
            "Versionable",
            "Publishable",
            "HasRelatedText"
          ],
        "isCheckedOut" : true,
        "lastModified" : "2013-10-28T14:27:36.84Z",
        "path" : "/asset-library/Screenshot from 2012-10-.1382970297489",
        "repository" : "default",
        "state" : "project",
        "title" : "Screenshot from 2012-10-29 12:26:10.png",
        "type" : "Picture",
        "uid" : "e9e1035f-0535-4aed-bb47-f5fa429bfc65",
        "versionLabel" : "0.0"
      },
      ...
      { "changeToken" : "1382970364262",
        "contextParameters" : { "documentURL" : "/nuxeo/site/api/v1/id/7f5af434-91ae-4d9c-b7f2-5bcd7a6427ce" },
        "entity-type" : "document",
        "facets" : [ "Picture",
            "Commentable",
            "Asset",
            "MultiviewPicture",
            "Versionable",
            "Publishable",
            "HasRelatedText"
          ],
        "isCheckedOut" : true,
        "lastModified" : "2013-10-28T14:26:04.26Z",
        "path" : "/asset-library/Screenshot from 2012-07-.1382970292974",
        "repository" : "default",
        "state" : "project",
        "title" : "Screenshot from 2012-07-07 17:09:39.png",
        "type" : "Picture",
        "uid" : "7f5af434-91ae-4d9c-b7f2-5bcd7a6427ce",
        "versionLabel" : "0.0"
      }
    ]
}

So we know how to query documents. But they have a lot of information that we don’t need and some that we need are missing. To have exactly what we want, we need to use the Business Object adapter(@bo).

Create a Slideshow Element Business Object

So let’s talk about business objects. It’s a way to adapt a document. When you are building a new API, you don’t want your users to have to deal with low level stuff. You want to keep it simple, to abstract the underlying plumbing of the Nuxeo repository as much as possible. This is why we have document adapters and business objects, to wrap a low level document into an object that is easy to use.

For my slideshow elements I will need the URLs of the thumbnail and the original image. I previously wrote a blog post on how you could do something similar using the RestContributorService extension point. While this is useful when you need something once, it’s not as good for re-usability. Using Business Object will fix this. Let’s write one.

A BO is a Java class that extends the BusinessAdapter class. It’s also a regular document adapter so you have to declare it as such, with its factory and XML contribution. It’s easy to do when you have Nuxeo IDE because we have a wizard for that. All the getters you declare here will be used by the JSONWriter when the response of your request is being built.

package org.nuxeo.slideshow;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.nuxeo.ecm.automation.core.operations.business.adapter.BusinessAdapter;
import org.nuxeo.ecm.core.api.ClientException;
import org.nuxeo.ecm.core.api.DocumentModel;
import org.nuxeo.ecm.platform.picture.api.adapters.PictureResourceAdapter;
import org.nuxeo.runtime.api.Framework;

/**
 * @author ldoguin
 */
public class SlideShowElement extends BusinessAdapter {

    private static final Log log = LogFactory.getLog(SlideShowElement.class);

    protected final DocumentModel doc;

    public SlideShowElement(DocumentModel doc) {
        this.doc = doc;
    }

    public String getId() {
        return doc.getId();
    }

    public String getType() {
        return doc.getType();
    }

    public String getTitle() throws ClientException {
        return doc.getTitle();
    }

    public String getDescription() throws ClientException {
        return doc.getProperty("dc:description").getValue(String.class);
    }

    public String getThumbnailURL() {
        PictureResourceAdapter picture = doc.getAdapter(PictureResourceAdapter.class);
        String blobPropertyName = picture.getViewXPath("Thumbnail") + "content";
        return getBigFileUrl(doc, blobPropertyName, doc.getName()
                + "_thumbnail" + ".jpg");
    }

    public String getOriginalURL() {
        PictureResourceAdapter picture = doc.getAdapter(PictureResourceAdapter.class);
        String blobPropertyName = picture.getViewXPath("OriginalJpeg")
                + "content";
        return getBigFileUrl(doc, blobPropertyName, doc.getName() + ".jpg");
    }

    private String getBigFileUrl(DocumentModel doc, String blobPropertyName,
            String filename) {
        String blobUrl = Framework.getProperty("nuxeo.url");
        blobUrl += "/nxbigfile/";
        blobUrl += doc.getRepositoryName() + "/";
        blobUrl += doc.getId() + "/";
        blobUrl += blobPropertyName + "/";
        blobUrl += filename;
        return blobUrl;
    }

}

<component name="org.nuxeo.slideshow.SlideShowElement">
   <extension target="org.nuxeo.ecm.core.api.DocumentAdapterService" point="adapters">
       <adapter class="org.nuxeo.slideshow.SlideShowElement"
           factory="org.nuxeo.slideshow.SlideShowElementFactory"/>
   </extension>
</component>

Once you deploy this, try the same query as before, but with the new adapter:

http://localhost:8080/nuxeo/api/v1/path/@search/@bo/SlideShowElement?query=Select * From Document WHERE ecm:mixinType = 'Picture'

The resulting JSON object will be simpler and contains the URLs we need:

{ "currentPageIndex" : 0,
  "currentPageSize" : 20,
  "errorMessage" : null,
  "hasError" : false,
  "isLasPageAvailable" : true,
  "isNextPageAvailable" : true,
  "isPaginable" : true,
  "isPreviousPageAvailable" : false,
  "isSortable" : true,
  "maxPageSize" : 100,
  "numberOfPages" : 7,
  "pageSize" : 20,
  "resultsCount" : 128,
  "entity-type" : "adapters",
  "entries" : [ { "entity-type" : "SlideShowElement",
        "value" : { "id" : "e9e1035f-0535-4aed-bb47-f5fa429bfc65",
            "originalURL" : "http://localhost:8080/nuxeo/nxbigfile/default/e9e1035f-0535-4aed-bb47-f5fa429bfc65/picture:views/item[4]/content/Screenshot from 2012-10-.1382970297489.jpg",
            "thumbnailURL" : "http://localhost:8080/nuxeo/nxbigfile/default/e9e1035f-0535-4aed-bb47-f5fa429bfc65/picture:views/item[3]/content/Screenshot from 2012-10-.1382970297489_thumbnail.jpg",
            "title" : "Screenshot from 2012-10-29 12:26:10.png",
            "type" : "Picture"
          }
      },
      ....
      { "entity-type" : "SlideShowElement",
        "value" : { "id" : "ff1eb80e-0fab-43e8-a886-38f739b78bef",
            "originalURL" : "http://localhost:8080/nuxeo/nxbigfile/default/ff1eb80e-0fab-43e8-a886-38f739b78bef/picture:views/item[4]/content/Screenshot from 2012-06-.1382970291187.jpg",
            "thumbnailURL" : "http://localhost:8080/nuxeo/nxbigfile/default/ff1eb80e-0fab-43e8-a886-38f739b78bef/picture:views/item[3]/content/Screenshot from 2012-06-.1382970291187_thumbnail.jpg",
            "title" : "Screenshot from 2012-06-13 14:56:41.png",
            "type" : "Picture"
          }
      }
    ]
}

Now we can retrieve all the information we want with a simple query. In the next blog post I’ll explain how I used this in an AngularJS app.

The post Creating a Thumbnail Browsing Module with AngularJS and the REST API – Part 1 appeared first on Nuxeo Blogs.

Yesterday I explained how you could retrieve documents through a search and adapt them with business objects, all using the REST API. Today I’ll show you how I used that to create a nice infinite scrolling page using AngularJS.

Start by forking nuxeo-angular-sample on GitHub. It’s a simple application that allows you to browse, edit, delete or create documents in a Nuxeo instance. Follow the readme to have it running; it’s quick and easy.

Now if you haven’t already, browse into the yo folder. The first thing I did was to modify the app/scripts/app.coffee file (yes, the sample is written in CoffeeScript). I added a new call to the when method in the $routeProvider object.

"use strict"
angular.module("nuxeoAngularSampleApp", ['nxSession','ui.bootstrap','blueimp.fileupload'])
.value("nxUrl", "/nuxeo/api/v1" )
.factory("nxSession", ["nxSessionFactory","nxUrl",(nxSessionFactory,nxUrl)->
  nxSessionFactory(
    apiRootPath: nxUrl
  )
])
.config ($routeProvider) ->
  $routeProvider
  .when("/nav/*path/edit"
    templateUrl: "views/edit.html"
    controller: "EditCtrl"
  )
  .when("/nav/*path/new"
    templateUrl: "views/edit.html"
    controller: "CreateCtrl"
  )
  .when("/nav/*path"
    templateUrl: "views/main.html"
    controller: "MainCtrl"
  )
  .when("/slideshow/:tag"
    templateUrl: "views/slideshow.html"
    controller: "SlideshowCtrl"
  )

  .otherwise redirectTo: "/nav/"

This means that each time someone hits the http://localhost:9000/#/slideshow/anyTag URL, he sees the content of the app/views/slideshow.html template included in the index.html file. This view is automatically bound to the SlideshowCtrl controller. So I need to create the app/views/slideshow.html file and add a new SlideshowCtrl controller.

As you probably guessed already, the HTML file goes into the app/views sub-folder. I’ll start by a very simple file containing only an h2 tag. The double braces are how you insert objects from your controller inside a template.

<h2>{{searchTag}}</h2>

If you go to http://localhost:9000/#/slideshow/anyTag, you will see {{searchTag}} and this is normal since we have not declared our controller yet. So to declare our controller, we will edit the app/scripts/controllers/main.coffee file. Just add this at the end:

.controller("SlideshowCtrl",
['$scope' ($scope) ->
  $scope.searchTag = 'hello world'
])

Now you should see a nice Hello World message on your page. It works because we’ve assigned the String value ‘hello world’ in the searchTag variable of our controller $scope object. As you can see we don’t need to instantiate the $scope. AngularJS injects it for us. Now what would be nice is to display the tag instead of the hello world message.

The when method call we added at the beginning was made on the $routeProvider object. Notice that all object from the AngularJS framework start with a ‘$’ char. We’re now going to inject this object just like we did for the scope.

.controller("SlideshowCtrl",
['$scope','$routeParams' ($scope,$routeParams) ->
  $scope.searchTag = $routeParams.tag
])

And as you can see it’s really easy to retrieve the tag segment of the URL. Now you should see something like this if you go to http://localhost:9000/#/slideshow/islands.

Remember that our goal is to display some documents resulting from a search. So we’re going to add a new service that will handle search. Let’s create a new file app/scripts/services/nxSearch.coffee to hold our search logic.

The code is pretty straightforward. Notice that we inject two AngularJS objects and one Nuxeo object: $http, $q and nxURL (see? no ‘$’ char there). nxURL does not come out of nowhere, it’s been defined in the app/scripts/app.coffee file.

angular.module('nuxeoAngularSampleApp')

.factory "nxSearch", ["$http","$q","nxUrl",($http,$q,nxUrl)->

  nxSearch = {}
  nxSearch.items = []
  nxSearch.busy = false
  nxSearch.isNextPageAvailable = true
  nxSearch.currentPageIndex = 0
  nxSearch.pageSize = 20
  nxSearch.query = undefined
  nxSearch.bo = undefined

  nxSearch.setBOAdapter = (bo)->
    nxSearch.bo = bo

  nxSearch.setPageSize = (pageSize)->
    nxSearch.pageSize = pageSize

  nxSearch.setQuery = (query)->
    nxSearch.query = query

  nxSearch.nextPage = ()->
    if !nxSearch.query?
      $q.reject("You need to set a query")
      return

    if !nxSearch.isNextPageAvailable
      return

    if nxSearch.busy
      return

    nxSearch.busy = true

    url = nxUrl + "/path/@search"
    if nxSearch.bo? then url += "/@bo/" + nxSearch.bo
    url += "?currentPageIndex="+nxSearch.currentPageIndex+"&pageSize="+nxSearch.pageSize+"&query=" + nxSearch.query;
    $http.get(url).then (response) ->
      docs = response.data
      if(angular.isArray(docs.entries))
        nxSearch.currentPageIndex = docs.currentPageIndex + 1
        nxSearch.isNextPageAvailable = docs.isNextPageAvailable
        nxSearch.items.push doc for doc in docs.entries
        nxSearch.busy = false
      else
        nxSearch.busy = false
        $q.reject("just because")

  nxSearch
]

This new CoffeeScript file will be automatically compiled to JavaScript by Grunt, but we still need to add the resulting JavaScript file to our code. Edit the index.html file to add this line at the end of the file:

    <script src="scripts/services/nxSearch.js"></script>

The next step is to use this code in our controller. Let’s start with something simple, like a list of documents:

    <ul>
      <li ng-repeat='item in items'>
        <span>{{item.title}}</span>
      </li>
    </ul>

Let’s modify our controller accordingly. I need to inject the nxSearch service and put an items variable in the scope.

.controller("SlideshowCtrl",
['$scope','$routeParams','nxSearch'
($scope,$routeParams,nxSearch) ->
  $scope.searchTag = $routeParams.tag
  # set our query
  nxSearch.setQuery("SELECT * FROM Document WHERE ecm:tag = '" +  $scope.searchTag + "'")
  # get the first result page
  nxSearch.nextPage()
  # expose our search items in the scope
  $scope.items = nxSearch.items
])

You should have something like this:

What about that infinite scroll I told you about? Because we only have the first page of results right now. Well it appears that there is an AngularJS module just for that. We’ll use Bower to install it. Simply type bower install nginfinitescroll --save while you’re in the yo directory. This will install the ngInfiniteScroll module and update the bower.json file where all the module dependencies are stored. We still have some work to do to enable that module. We need to add it to our module configuration in the app/scripts/app.coffee file.

angular.module("nuxeoAngularSampleApp", ['nxSession','ui.bootstrap','blueimp.fileupload','infinite-scroll'])

And we also need to include the JavaScript library at the end of our index.html file:

    <script src="scripts/services/nxSearch.js"></script>

Now we can start using the module. Go to your template and modify the <ul> tag like this:

<h2>{{searchTag}}</h2>

    <ul infinite-scroll='nextPage()' infinite-scroll-disabled='busy' infinite-scroll-distance='1'>
      <li ng-repeat='item in items'>
        <span>{{item.title}}</span>
      </li>
    </ul>
  <div ng-show='busy'>Loading data...</div>

</div>

This won’t work as is, we need to expose the busy variable and the nextPage method from our scope. Now we can modify our controller like this:

.controller("SlideshowCtrl",
['$scope','$routeParams','nxSearch'
($scope,$routeParams,nxSearch) ->
  $scope.searchTag = $routeParams.tag
  nxSearch.setQuery("SELECT * FROM Document WHERE ecm:tag = '" +  $scope.searchTag + "'")
  $scope.items = nxSearch.items
  # Expose the busy variable
  $scope.busy = nxSearch.busy
  # Expose the nextPage method
  $scope.nextPage = ()->
     nxSearch.nextPage()

])

And that’s it. You now have infinite scrolling of a document search. In the next part I’ll show you how to handle the Business Object adapter and show a nice thumbnail grid :-)

The post Creating a Thumbnail Browsing Module with AngularJS and the REST API – Part 2 appeared first on Nuxeo Blogs.

We just released Nuxeo 5.8, the first Long Term Support version made on our new release cycle. This is the version you should use for production now – it will be supported for the next three years. If you want to know what has changed between 5.7.3 and 5.8, please take a look at the release notes. If you want to know what has changed between 5.6 and 5.8, I have merged all the release notes in between those two.  And if you want to follow what has changed between each version of Nuxeo, you should take a look at the release notes index.

If you haven’t followed the latest Fast Track releases, our main focus has been to improve our REST API, DAM module, Nuxeo Drive, Workflow engine and UX. Among the new features available since the latest fast track, 5.7.3, is a task resolution tab. It’s a dedicated screen to solve one of your tasks and stay focused on the workflow. This can be helpful, for instance, with the Parallel Review workflow we just added.

Some of the feedback we received during Fast Track development related to issues people had when using Drive behind a proxy. So now you can add your proxy settings directly in a dedicated panel in Drive configuration.

Something else that should catch your attention when you try Nuxeo Platform 5.8 is the new look and feel. Lise has changed the default color flavor and updated all the icons. It gives Nuxeo a cleaner, modern look.

On a totally different topic, we’ve integrated Redis with Nuxeo. It’s an optional feature that can be easily configured in nuxeo.conf. All you have to do is use the dedicated RedisService. The first one to benefit from it is the WorkManager. If you use the Redis implementation, you’ll gain:

• High-Activity work queueing, to avoid saturating memory or blocking
• Queue persistence, to avoid lost jobs on shutdown
• New feature: node-aware job execution

Other topics include updates in the PDF signature module, Content Automation and the REST API. We also did some work on our test framework. We integrated Mockito to ease Unit Testing and improved our WebDriver tests. Also, if you are a blog or website user, know that they have been removed from DM and are now an independent marketplace package.

A big Thank You to all the contributors, whether it was code, localization, documentation, tests, feedback. Thanks to Inigo Surguy, Jordi Mallach, Marwane Kalam-Alami, Nelson Silva, Patrick Buschman, Patrick Turcotte, Ron, Stéphane Galland, Sun Tan, Sylvain Chambon, Toweins, Vincent Bonamy for helping us make this new version of Nuxeo Platform. Sorry if I’ve forgotten anyone.

You can download Nuxeo Platform 5.8 from our website, or attend the live webinar on November 6 to see a demo.

What’s next you might ask? Well we have already started working on the next Fast Track Nuxeo Platform 5.9.1. I’ll keep you posted on the scope of this release.

The post Nuxeo Platform 5.8 Long Term Support Release appeared first on Nuxeo Blogs.

Nuxeo Studio 2.15 was officially released last week, and it comes with Nuxeo Platform 5.8.  In this version of Studio we expose some of the latest capabilities added in the platform, providing more configurability. This includes:

• DAM widgets are now available in Studio. You can use them in tabs you create: Video story board, video reader, picture web view….

• All Select2.js widgets have been exposed in Studio, with all their options. Select2.js widgets are the suggestion widgets in the platform. You have suggestion widgets in simple selection and multi-selection objects for users (silently replacing previous implementation), directories, documents (silently replacing previous implementation), and there is an advanced one to leverage any operation which returns some correctly formatted JSON, for advanced integration. Those widgets present new interesting options, such as being able to define the NXQL query used inline, or being able to format the value displayed in the list, thus making it possible to display multiple fields of the document in each drop-down item. Those widgets are a great illustration of the maintainability of the platform. Users who configured their forms in Studio two or three years ago will benefit from this new set of widgets without any effort, as they replace the former implementation, when relevant.

• New categories of actions have been added to the header and the footer. You can use the “Main Tabs” category for adding new navigation links to strategic locations in your repository (links will appear beside the Home and Document Management tabs).
• New operations are exposed.
• Faceted search configured with Studio that uses full text search is now compliant with the top-right suggest box: if you click from there on the full-text search, provided you have a full text search field on your faceted search, everything will work smoothly.
• Default Application templates for DM and DAM have been updated.
• A new parameter in the flavors has been added, so you ‘ll have to set a color for this one if you don’t want the default “blue” that appears on the selected main tab.

Finally, we took your feedback into account, and now better manage the fast track transition. There is still room for improvement, but now, when you connect and your project is on a Fast Track version that is not supported anymore by Studio (e.g. 5.7.2), Studio forces you to go the target version setting screen where you can set an existing value (you may want to migrate to 5.8 currently).  Also note the new download link beside the target platform making it easy for you to get the correct version of Nuxeo!

We hope you will like this new release and make great use of it! For the next few months we plan to focus on usability of existing features, putting an emphasis on the user experience. We do not plan to add too many new capabilities. We will start with improving the application templates feature, a key piece for bootstrapping your configuration with some existing templates, making Studio easier to use.

The post Nuxeo Studio 2.15 Has Been Released appeared first on Nuxeo Blogs.