Open topic with navigation

Using Solr search

Solr is an open-source, enterprise-level search platform, from the Apache Lucene project, that is known for scalability and performance. In its implementation of Solr, Ektron uses Apache Tomcat to host the Solr application, and ManifoldCF to manage the crawling of new or updated content. According to https://lucene.apache.org/solr/, "Solr powers the search and navigation features of many of the world's largest internet sites."

Using Solr, you can search all Ektron content, including Smart Forms and assets. Ektron's Solr implementation supports advanced search features, such as Refinements, Synonym Sets, Suggested Results, and Autocomplete. Solr also supports Ektron's templated search server controls and API. So if you previously worked with these components, you only need to master a few new concepts to migrate to Solr.

NOTE: To compare Solr with other Ektron search providers, see Search provider comparisons .

Solr Search Components

The Ektron Solr search provider is made up of several components, which can crawl your content to index the latest changes, as well as manage search queries. Solr manages the search for any number of Ektron instances, each of which can support several sites.

Ektron crawl service

This diagram explains the crawl service's activities.

The Apache ManifoldCF (connector framework) crawls data from a repository connector to an output connector. Ektron's Solr deployment uses its own repository connector with the Solr output connector that updates the Solr Index.

When a site is registered, the following steps occur within ManifoldCF.

  1. An Ektron repository connection is created, using database information from the Ektron site.
  2. A Solr output connection is created with a connection to a Solr core exclusive to the Ektron site.
  3. Content, user, and community group jobs are created for each Ektron repository/Solr output connection pair to crawl the various Ektron items.

Asset transfer service

Ektron's crawl component uses the Asset Transfer Service as the endpoint Part of a DXH connection, it is the combination of an endpoint (a URL or API path) and authentication credentials that allows communication between 2 software instances. You define DXH endpoints in your site’s web.config file. for crawling assets. The Ektron Asset Transfer Server, which resides on the Ektron Web server, is one half of a pair of services that supports the indexing of assets. (The Ektron Asset Transfer Client resides on the Solr server.) When an asset is indexed, it must be transferred to a cache on the Solr server before its content can be extracted.

When the crawl service requires an asset, it asks the Ektron Asset Transfer Client to retrieve it. If the Ektron site and Solr run on the same server, the client directs the indexer to the asset's location on the file system. If the asset resides on a remote server, the client requests the asset from the appropriate Ektron Asset Transfer Server and stores it in cache for indexing.

The crawl database

The crawl database stores and maintains the crawl state and crawl queues during a crawl.

Ektron Solr admin service

The Ektron Solr Admin service manages

  • site registration requests (from the Solr Admin Console)
  • crawl management requests (from Ektron and EWS Ektron Windows Service during eSync)
  • status requests from the Ektron Workarea's Search Status screen

During site registration, the admin service performs

  • validation tasks prior to site registration
  • job deletion and creation in ManifoldCF
  • Solr core deletion and creation
  • search configuration store and Ektron database updates with registration information

the Solr Admin service also communicates with the crawler component (ManifoldCF) of the Ektron site and fires an appropriate crawl of jobs for the core.

Ektron query services

This diagram depicts IIS query services that act as a proxy to the Solr counterparts.

The Query Service routes queries from sites registered with the Solr Search server to the Solr core that corresponds to the site.

The Query Proposition service is a RESTful service that routes autocomplete queries to the Solr core that corresponds to the site.

The Search Config database serves as a "centralized store." It maintains configuration information for search endpoints, Solr, ManifoldCF, Admin Service, and registration information for the registered sites.

Solr core

This diagram details the Solr core, the rectangle on the right side of the above image.

A Solr core is a single instance of Solr with its own configuration, schema, and independent index. Each Solr core can be thought of a compartmentalized Solr application. It has several endpoints, called handlers, that serve requests addressed to the core.

Ektron's search deployment uses handlers to manage queries (search queries & completion requests), administrative requests (to manage cores), and content updating.

Mime types supported by Solr search

Solr's ExtractingRequestHandler uses Tika to lets users upload binary files to Solr, and then have Solr extract text from and index them. For a list of supported file types, see https://tika.apache.org/1.4/formats.html.

Also, Ektron has its own list of files types that may be uploaded to the Workarea. You can only upload supported file types. See Supported types of assets.

See also: Excluding Library Images When Using Solr Search.

Solr search concepts

This section explains some inner workings of the Solr search. It is intended to help explain why you do or do not get certain search results.

Keyword search

Keyword search isthe search performed when a user types text or a phrase into a text field. (A phrase is several words enclosed in quotes. For example, “This is a phrase”). Keyword search is performed against the content title and the body only, because they contain the "meat" of a content item.

NOTE: For HTML content, content body corresponds to the content table's content_html column. For PageBuilder pages, content body corresponds to the content table's content_text column.

Content title and body are subject to linguistic processing, which applies rules based on the language of the search text or phrase. (By default, this is the site language, but it can be changed via the Search API). For Solr-based search, linguistic processing refers to stemming and stop word recognition only.

Stemming

Stemming reduces a word to its stem. English examples of stemming:

  • engineering > engine
  • parsing > pars
  • analysis > analysi

Stemming helps a user find desired content because, in most cases, the user is unsure of the exact text in the document.

Stemming is applied when content is indexed and when query terms are processed. During indexing, the content language is used to perform language-specific stemming. During querying, the language of the query (by default, the site language) is used to perform stemming.

Sometimes, stemming produces false matches. For example, the terms "engineering" and "engine" are both reduced to "engine" during indexing. So, a query for "engine" matches a document that contains "engineering". Also, since Solr only performs stemming reduction, a query containing "ran" does not match a document containing "running", since these words get reduced to "ran" (query side) and "run"(index time). Out-of-the-box Solr only supports stemming and not the more extensive lemmatization, which provides more accurate results and fewer false positives.

Stop words

Stop words are terms that are removed from queries and indexes because they do not contribute to the quality of a search. For example, these are English stop words: "the", "in," "an", "a".

Query expression

The QueryText property of the KeywordSearchCriteria class contains the text or phrase that a user enters when submitting a keyword search. With regards to Solr, any query expression assigned to the ExpressionTree property of the KeywordSearchCriteria class is treated as a filter query. The filter query merely helps in reduction of the result subset and does not improve relevance.

Rank computation (relevance)

When ranking content returned as search results, Solr uses the quality of a document's match against query text. Match quality is based on these factors.

  • Frequency of search text or phrase in the content.
  • Rarity of search terms with regards to the whole corpus of content (the entire site).
  • Proximity of search terms in the content. For example, if a query has the phrase "content repository", content having "The content item rocks" in closest proximity is ranked higher than content with the terms "lack of actual content in an item."
  • If search terms include an OR operator, content with the greatest number of terms ranks highest. For example, if a query consists of "valid OR resultant OR set OR items", content with "valid resultant set" ranks higher than content with "resultant set."

Notes regarding rank computation (relevance)

  • When computing rank, Ektron Solr only considers terms in the content's title and body.
  • Only terms entered against the QueryText property of the KeywordSearchCriteria class are used for rank computation. Other fields specified in the ExpressionTree property are not considered for rank computation.
  • A match on content title has the same weight as a match on content body.

ContainsExpression vs EqualsExpression

When using Solr, the ContainsExpression can be tricky.

NOTE: A ContainsExpression involves using ContainsExpression directly or indirectly via SearchContentProperty.<PropertyName>.Contains.

Unlike the C#/.NET String.Contains, Solr's ContainsExpression matches whole words only—it does not match on substrings. For example, if a text metadata field contains "Tax Revenue Statement, Year 2012," a contains search returns the content if a query using ContainsExpression includes "Tax" or "Revenue Statement" or "Revenue" as whole words. The query is not a match for the substring term "State."

Also, the ContainsExpression query is case-insensitive. So for example, "Revenue" is the same as "revenue" and "rEvenUe."

The EqualsExpression, on the other hand, performs a letter to letter, case-sensitive, exact search on the entire text of a field.

NOTE: An EqualsExpression involves either EqualsExpression directly or indirectly via SearchContentProperty.<PropertyName>.EqualTo.

To continue the above example for an EqualsExpression, a match results only if the query is submitted as “Tax Revenue Statement, Year 2012”. Note the comma in the query.

Using .Contains with Solr in Ektron version 9.30

When using the Solr search provider in Ektron version 9.30, the FolderId, FolderPath, and TaxonomyPath .Contains syntax acts like a startswith expression instead of a Contains one. Examples:

FolderIdPath examples
Taxonomy examples
Exception

Configuring the Ektron Solr package for Java Heap Memory

The Ektron Solr package has the Java Heap Memory (HSQLDB-Crawl Database, ManifoldCF-Crawler, and Tomcat-Solr Process) tuned to support the simultaneous registration and crawling of multiple sites. This tuning is optimized for a server with at least 8GB RAM and 2 high speed disks (1 for the search index, and 1 for search installation). The tuning also accommodates servers with more than 8GB RAM.

Because the Solr configuration is less reliant on the Java Heap for loading the index , the default tuning should suffice for most production usages. Before changing the Java Heap configuration, Ektron's Professional Services Group must evaluate your plans to ensure that correct choices are being made with regards to garbage collection patterns and search/crawl performance.

After a site is registered, the first query served by Solr is slower than subsequent ones. If the Solr server is rebooted, you can expect similar slowness for registered sites. To avoid this slow first response, create a dummy search request to be run after a site is registered for the first time. You can also avoid this slow first response by issuing a search request from a site (for every site) after the Solr search server has been rebooted.

You may perform additional performance tuning via the Apache Solr site. Before making any adjustments, you must have a thorough understanding of the technologies. Be very careful when tuning for performance.

Query suggestions vs query completions

Query suggestions are a search type-ahead feature, in which a search field presents query options as the user begins to type in characters. Suggestions are based on previously used, successful queries submitted by other users.

Query completions are another search type-ahead feature in which the search field presents query options as a user begins to type in characters. However, in this case, query options are based on either the search index (the default option) or a dictionary of terms.

Ektron Solr only supports query completions.

You can change the source of query completions from the search index to a dictionary of terms. See also: Managing Solr autoComplete.

With regards to query completions, the search only considers terms from the title by default. You can use a custom field instead of title. To accomplish this, customize the configuration during registration. See also: Part 2: Register your Solr site.

Securing the Solr server

To maximize the security of your Solr server, follow these guidelines.

Setting up Solr search

Setting up Solr search on a single on-premises (local) server

  • For the configuration database, SQL 2008 or higher (Express version is supported); can be installed to any server, for example the Ektron server, the Solr server, or any other database server.

Preparing the Ektron server to work with Solr search

  1. On the Ektron search server's firewall, open port 8732. This runs Ektron Windows service and asset transfer server services.

  2. Restart the Ektron server to apply all changes, not only those affecting SQL but also the JDK installation, new environment variables, and so on.

NOTE: You must generate an eSync certificate to start Ektron Windows Service to register a site.

Preparing the Solr search server to work with Ektron

During Solr installation, the user selects a base port (the default is 7600). The installation allocates that port and nine above it to accommodate Solr search. For example, if you use the default value of 7600, ports 7600 through 7611 are dedicated to Solr search.

On the Solr search server's firewall, open to the Ektron Web server ports +0 and +1 in the range—they are used by Ektron’s query and admin services. To continue the default example, open ports 7600 and 7601.

Prerequisites

  • Solr server
    • Hardware requirements: minimum 8 gigabytes of RAM, quad core, 64-bit processor
    • Operating system: Windows 2008, Windows 2012, Windows 2016

      NOTE: For a production server, use Windows 2008 R2, Windows 2012, or Windows 2016.

    • Enable the following
      • Roles
        • Application Server Role Services
          • .NET Framework 4.5

            IMPORTANT: With Windows Server 2008 or 2008 R2, be sure that .NET 3.5 is installed before installing 4.5.

          • Web Server IIS Support
        • Web Server (IIS) Role Services
          • Windows Authentication feature
          • .NET Extensibility 4.5
          • ASP.NET 4.5
      •  Features
        • .NET Framework > WCF Activation

          NOTE: Make sure that HTTP activation is also checked.

        • Static content
      • SQL browser, if not using default instance
    • For optimal performance, use a separate disk (that is, not the system drive) for index and asset cache
    • May be installed on Ektron server but only in a development environment.
      Consult an Ektron certified partner to determine your server requirements. Ektron does not recommend installing Solr on a server that also hosts Microsoft Search Server.
    • Java Development Kit 1.7, Windows x64 version.

      To download, go to https://www.oracle.com/java/technologies/javase/javase7-archive-downloads.html.

      IMPORTANT: If you upgrade the JDK version, you must replace it in the same directory to which you originally installed it. You may not install more than one JDK version to that directory.

  • An Ektron site installed on an Ektron server. See also: Installing Ektron, Upgrading to Ektron 9.10 SP3.

    NOTE: During the Ektron installation, you should have selected Solr as your search provider. If you need to change your search provider to Solr, see Changing your search provider to Solr.

  • The Ektron Solr installation file. Contact your Ektron account manager to obtain a link to the file.
  • Users
    • A SQL login account assigned the dbcreator role. This account is used to create the search configuration database.
    • A SQL login account that Ektron search components use to access the search configuration database.

      Both SQL accounts must follow these rules:

      • If integrated security is set to true

        ...and Solr search and the Ektrondatabase are on different servers, a domain user is required.

        ...and Solr search and the Ektron database are on the same server, a local Windows user is required.

      • if integrated security is set to false, a SQL server user.

      Best Practice 

      Use SQL authentication rather than Integrated Security. Although Integrated Security is supported, the connection is made in the context of the search service account (Windows user) provided during installation. If this user does not have access to the database, the connection fails.
    • A Windows user account under which Solr services will run. This account, which manages Solr components and authenticates queries, must have permission to
      • write to the search 2.0 folder and its subfolders
      • start the Solr admin service
      • log on as a service

      This user can be either a Windows user on the search server, or a domain user. If it is a domain user, the Netlogon service must be enabled and running.


  • Knowledge of Solr, for administrators who want to perform advanced tasks

Installing Solr on a single on-premises (local) server

After installing Ektron, you can install Solr. This is a 2-step procedure.

  1. Install files needed to run Solr.
  2. Register the Ektron site with the Solr search engine.

See also: Using Solr search with 3-tier architecture.

Part 1: Run Solrinstall.exe
  1. Contact your Ektron account manager to obtain a link to download the Solr installation file.
  2. Log in as a Windows Administrator user.
  3. Download and run the Ektron Solr installation file as an administrator.
  4. Accept the license agreement.

  5. Select a folder in which to save the Solr installation files.

  6. You are prompted to select a base port. After you select a port, it and 9 consecutive ports are designated for Solr use. For example, if you accept the default port 7600 as the base, ports 7600 through 7611 are used for Solr.

    NOTE: Do not use a port number below 1025. Also, do not use ports 6080, 6081, and 8732: they are reserved for other Ektron components.

  7. Enter credentials (domain, user name, and password) of the Windows user account under which Solr query services will run. This account manages Solr components and authenticates queries. The account must have permission to
    • write to the search 2.0 folder and its subfolders
    • start the Solr admin service
    • lock pages in memory. You can assign this in the machine's Local Security Policy under Local Policy > User Rights Assignment.
    • log on as a service. If the user does not have "Log on as a Service" right, you are prompted to assign it.

    If you are using a domain account, the Netlogon Service must be enabled and running.

    `

  8. Enter the SQL Server that will host the Solr configuration database. Then either select Trusted Connection or enter the username and password of the user who will create the search configuration database. The user must be assigned the SQL Server dbcreator role.

  9. Enter a unique name for your search configuration database. Spaces are not allowed.

  10. Enter credentials for a SQL login account that Ektron search components will use to access the search configuration database.

  11. Enter the host name of the server on which you are running the Solr installation. The Ektron server must be able to access this host.

  12. Select a folder in which Solr will store its index data.

  13. Specify the location of your asset cache. The asset cache is used only if the Ektron asset files are not stored on your Solr search server.

  14. A summary of installation choices appears. If everything looks OK, click Install.

    Key points on this screen:

    • The query service port is always the base port.
    • The screen lists your search service account, configuration database, and the port number assigned to the following:
      • Query service. Outbound from Ektron server
      • Admin service. Outbound from Ektron server
      • Tomcat connector. For local communications between Solr and Tomcat
      • Tomcat shutdown. For local communications between Solr and Tomcat
      • ManifoldCF. For local communications between Solr and ManifoldCF
      • Asset Transfer Client. Two-way between Ektron and Solr servers

    IMPORTANT: The summary screen's first 2 ports have an asterisk (*). Set up the Ektron server's firewall such that these ports let the Ektron site communicate with the Solr service.

Part 2: Register your Solr site

  1. Remote onto the Solr machine and go to the following URL: https://YourSolrMachineName:7600/SolrAdminConsole/Index.html.
  2. If prompted, enter the credentials of the search server user used to install Solr. If you are uncertain of the username, go to the Windows Services panel, locate the Ektron Solr Process Manager service, and identify the user in the Log On As column. Use this user to login to the site listed in step one.
  3. In the Web Server Name field, enter the machine name of the server that contains the CMS site you are registering and click Connect.
  4. A list of Ektron databases on the server should be listed. Identify the database you wish to register.
  5. If you deleted the crawl database, click the Start over button, confirm that the crawl filter and polling interval settings are correct, and hit the Register button. Take time to consider the crawl filters-your choice can make a considerable difference in crawl time.

    If you are registering the site for the first time, the steps are the same minus the start over step.
  6. If everything is configured correctly, you see a registration successful message. Otherwise, search the knowledge base for the error you are receiving. If necessary, contact Support.

Setting up Solr search in the Amazon cloud

This section explains how to set up Solr search to support an Ektron website deployed to the Amazon cloud.

Part 1: Set up an Ektron site in the Amazon cloud

When prompted to select a Web Instances Type, select m1.large.

Part 2: Create a security group

The security group authorizes the opening of ports, which access search components between Ektron and the Solr VM. Two of the ports, selected during the Solr installation, are base port and base port+1. By default, they are 7600 and 7601.

NOTE: The base port is for the Admin Console/Registration UI and Query services. The base+1 port is for the Ektron Solr Admin Service.

NOTE: Create the Security Group for the region you selected for the Cloud site (for example, US East).

  1. Sign in to the Amazon Web services Ektron Windows Service uses Windows Web Services to perform these activities: • Schedules the future publication and removal of content. • Transmits notifications that a sync should be started between the staging and production servers. • In the 3-Tier feature, transfers data between servers. console.
  2. Click Services > EC2 > Security Groups > Create Security Group.

  3. Add a Name, Description and select No VPC.

  4. Click the group and add these 2 inbound rules.
    Rule: TCP
Port range: 3389
Source: 0.0.0.0/0
    Rule: TCP
Port range: 7600-7601
Source: 0.0.0.0/0

    NOTE: For the second inbound rule, you are not required to use ports 7600 and 7601. These are only examples of ports you might use.

  5. Click Apply Rule Changes.

Part 3: Create Solr search VM

In this part, you create a Solr search version on an Amazon virtual machine.

NOTE: Create the Solr Search VM in the same region you selected for the Cloud site.

  1. Sign in to Amazon Web services Ektron Windows Service uses Windows Web Services to perform these activities: • Schedules the future publication and removal of content. • Transmits notifications that a sync should be started between the staging and production servers. • In the 3-Tier feature, transfers data between servers. console.
  2. Click EC2 > Instances > Launch Instance.
  3. Click the Quick Start tab.
  4. Select a Microsoft Windows server that supports Solr search. Do not choose one that is bundled with a SQL server. See also: Setting up Solr search on a single on-premises (local) server

  5. Click the Select button. A new window appears.
  6. Select All Instance Types > M1 large and click Next.

  7. On the Configure Instance screen, check Protect against accidental termination and click Next.

  8. The Storage Device Configuration screen appears. Click Next. The Tag Instance screen appears.
  9. Add a tag. For the Key, enter search. For the Value, enter solr. Click Next.

  10. Select the security group you created in Part 2: Create a security group and click Review and Launch.

  11. Review screen information then click Launch.
  12. Select the key pair that you created and click Launch Instances.
    If you created a new key pair when you ran the deployment wizard, the CloudPemFile is created in the Ektron on-premises Windows Service install directory, C:\Program Files (x86)\Ektron\EktronWindowsService40\CloudPemFile.

    NOTE: If you do not have an existing key pair, create new pair and save the key pair file on your computer. You will need it later to log in.

  13. Move the cursor to the row containing the new instance, click the Name field, then insert a name.

    NOTE: This step is not required but will ease identification of your VM later.

  14. When the launch is finished, right click the instance and select Get Windows Password.

  15. Browse to the keypair file described in Step 10, EktronSolrSearchKP.pem, and click Decrypt Password.

  16. You see the decrypted password & hostname. Save them for future access to the EC2 Instance Remote Desktop Session.

Part 4: Set up the Amazon Solr search VM

On the Solr search Amazon VM (which you installed in Part 3: Create Solr search VM), install the Solr search prerequisites.

Prerequisites

  • Solr server
    • Hardware requirements: minimum 8 gigabytes of RAM, quad core, 64-bit processor
    • Operating system: Windows 2008, Windows 2012, Windows 2016

      NOTE: For a production server, use Windows 2008 R2, Windows 2012, or Windows 2016.

    • Enable the following
      • Roles
        • Application Server Role Services
          • .NET Framework 4.5

            IMPORTANT: With Windows Server 2008 or 2008 R2, be sure that .NET 3.5 is installed before installing 4.5.

          • Web Server IIS Support
        • Web Server (IIS) Role Services
          • Windows Authentication feature
          • .NET Extensibility 4.5
          • ASP.NET 4.5
      •  Features
        • .NET Framework > WCF Activation

          NOTE: Make sure that HTTP activation is also checked.

        • Static content
      • SQL browser, if not using default instance
    • For optimal performance, use a separate disk (that is, not the system drive) for index and asset cache
    • May be installed on Ektron server but only in a development environment.
      Consult an Ektron certified partner to determine your server requirements. Ektron does not recommend installing Solr on a server that also hosts Microsoft Search Server.
    • Java Development Kit 1.7, Windows x64 version.

      To download, go to https://www.oracle.com/java/technologies/javase/javase7-archive-downloads.html.

      IMPORTANT: If you upgrade the JDK version, you must replace it in the same directory to which you originally installed it. You may not install more than one JDK version to that directory.

  • An Ektron site installed on an Ektron server. See also: Installing Ektron, Upgrading to Ektron 9.10 SP3.

    NOTE: During the Ektron installation, you should have selected Solr as your search provider. If you need to change your search provider to Solr, see Changing your search provider to Solr.

  • The Ektron Solr installation file. Contact your Ektron account manager to obtain a link to the file.
  • Users
    • A SQL login account assigned the dbcreator role. This account is used to create the search configuration database.
    • A SQL login account that Ektron search components use to access the search configuration database.

      Both SQL accounts must follow these rules:

      • If integrated security is set to true

        ...and Solr search and the Ektrondatabase are on different servers, a domain user is required.

        ...and Solr search and the Ektron database are on the same server, a local Windows user is required.

      • if integrated security is set to false, a SQL server user.

      Best Practice 

      Use SQL authentication rather than Integrated Security. Although Integrated Security is supported, the connection is made in the context of the search service account (Windows user) provided during installation. If this user does not have access to the database, the connection fails.
    • A Windows user account under which Solr services will run. This account, which manages Solr components and authenticates queries, must have permission to
      • write to the search 2.0 folder and its subfolders
      • start the Solr admin service
      • log on as a service

      This user can be either a Windows user on the search server, or a domain user. If it is a domain user, the Netlogon service must be enabled and running.


  • Knowledge of Solr, for administrators who want to perform advanced tasks

Part 5: Install Solr on the Solr VM

Remote into your Solr VM then complete these steps.

  1. Contact your Ektron account manager to obtain a link to download the Solr installation file.
  2. Log in as a Windows Administrator user.
  3. Download and run the Ektron Solr installation file as an administrator.
  4. Accept the license agreement.

  5. On the Deployment Type screen, choose Install Solr on a single server.

  6. Select a folder in which to save the Solr installation files.

  7. You are prompted to select a base port. After you select a port, it and 9 consecutive ports are designated for Solr use. For example, if you accept the default port 7600 as the base, ports 7600 through 7611 are used for Solr.

    NOTE: Do not use a port number below 1025. Also, do not use ports 6080, 6081, and 8732. They are reserved for other Ektron components.

  8. Enter credentials (domain, user name, and password) of the Windows user account under which Solr query services will run. This account manages Solr components and authenticates queries.

    The account must have permission to

    • write to the search 2.0 folder and its subfolders
    • start the Solr admin service
    • log on as a service. If the user does not have "Log on as a Service" right, you are prompted to assign it.

    If you are using a domain account, the Netlogon Service must be enabled and running.

    `

  9. Enter the SQL Server that will host the Solr configuration database. Then either select Trusted Connection or enter the username and password of the user who will create the search configuration database.

    To get the SQL server name, use the Endpoint field in the Amazon Cloud Instances screen. Exclude the Endpoint's port number.

    The user must be assigned Create Any Database permission on the RDS database server. See also:  Amazon Relational Database Service (RDS) documentation.

  10. Enter a unique name for your search configuration database. Spaces are not allowed.

  11. Enter credentials for a SQL login account that Ektron search components will use to access the search configuration database.

  12. Enter the host name of the server on which you are running the Solr installation. The Ektron server must be able to access this host. Your host name may look like this.

    The following image indicates where to get the Amazon load balancer host name.

  13. Select a folder in which Solr will store its index data.

  14. Specify the location of your asset cache. The asset cache is used only if the Ektron asset files are not stored on the Solr search server.

  15. A summary of installation choices appears. If everything looks OK, click Install.

    Key points on this screen:

    • The query service port is always the base port.
    • The screen lists your search service account, configuration database, and the port number assigned to the following:
      • Query service. Outbound from Ektron server
      • Admin service. Outbound from Ektron server
      • Tomcat connector. For local communications between Solr and Tomcat
      • Tomcat shutdown. For local communications between Solr and Tomcat
      • ManifoldCF. For local communications between Solr and ManifoldCF
      • Asset Transfer Client. Two-way between Ektron and Solr servers

    IMPORTANT: The summary screen's first 2 ports have an asterisk (*). On the Ektron server, adjust the firewall to allow these ports to communicate between your Ektron site and the Solr service.

Part 6: Set up a search load balancer

In this section, you load balance your search. You need to do this so the cloud-based Ektron site can access search components on the Solr VM, which will be accessible through the load balancer.

  1. From the Amazon Web services Ektron Windows Service uses Windows Web Services to perform these activities: • Schedules the future publication and removal of content. • Transmits notifications that a sync should be started between the staging and production servers. • In the 3-Tier feature, transfers data between servers. console, go to EC2 > Load Balancers > Create Load Balancer.
  2. Give the Load Balancer a name.
  3. At Create LB Inside, choose EC2.
  4. Map the base and base + 1 ports (by default, 7600 and 7601) between load balancer and Instance. Use TCP protocol, not Http. Click Continue.

  5. On the Health Check dialog
    • at Ping Protocol, select HTTP.
    • at Ping Port, enter the base port. By default, it is 7600.
    • at Ping Path, enter /ping.html.

  6. The Add EC2 Instances screen appears. Select the instance you created in Part 3: Create Solr search VM and click Continue.
  7. A review screen displays the settings. When you are ready, click Create. Amazon sets up a Search Load Balancer.

    NOTE: The health check may fail since Solr Install has not run yet, and ping.html is not present on the VM.

Part 7: Edit health check, create inbound rule, and verify QueryService URL

  1. From the Amazon Web services Ektron Windows Service uses Windows Web Services to perform these activities: • Schedules the future publication and removal of content. • Transmits notifications that a sync should be started between the staging and production servers. • In the 3-Tier feature, transfers data between servers. console, go to EC2, click Load Balancers, and select the load balancer you created.

  2. On the Amazon Solr server, create an inbound rule to allow connections to Solr ports.This allows your Ektron VM to communicate with the Solr Admin Console. To do this:
    1. Go to Control Panel > Windows Firewall > Advanced settings > Inbound Rules > New Rule.

      NOTE: Leave firewall status "Turned On".

    2. On the Rule Type screen, choose Port.
    3. On the Protocols and Ports screen, choose TCP. In the Specific local ports field, enter your baseport and baseport + 1. For example, 7600-7601.
    4. Click Next then Allow the connection.
    5. On the When does this rule apply? screen, check all boxes.
    6. Assign a name to the rule.
  3. Click the Health Check tab. Click Edit Health Check. Change the value of ResponseTimeout to any value and save. Verify that this action changes your load balancer status to In Service.
  4. On the Amazon Ektron server, create an inbound rule to add 8732 as an inbound port. This allows the Solr Admin Console to communicate with your Ektron VM. To do this:
    1. On the Amazon instance that hosts Ektron, go to Control Panel > Windows Firewall > Advanced settings > Inbound Rules > New Rule.

      NOTE: Leave Firewall status "Turned On".

    2. On the Rule Type screen, choose Port.
    3. On the Protocols and Ports screen, choose TCP. In the Specific local ports field, enter 8732.
    4. Click Next then Allow the connection.
  5. From Amazon EC2 console, obtain the DNS Name of the search load balancer.

  6. Verify that the QueryService Url works from the Ektron server via this URL: https://load balancer DNS name:baseport number/SolrQueryService/SolrQueryService.svc

Part 8: Disable "Set Computer Name"

  1. On the Solr search Amazon VM, press Windows key\S to open the search box and enter EC2ConfigService.
  2. On the Ec2 properties dialog, uncheck the Set Computer Name box.

Part 9: Register the load-balanced site

Register the load balanced site using the following URL: https://load balancer DNS name:baseport number/SolrAdminConsole/index.html.

Additional steps if Ektron is installed manually on Amazon EC2

If Ektron was manually installed in an Amazon EC2 environment, and you want Solr search to crawl Ektron resource files, follow one these procedures to enable that crawl.

Store resource files on the EC2 instance

Follow these steps to allow Solr search to crawl Ektron resource files (assets, private assets, uploaded images, uploaded files, asset library) on an EC2 instance. To properly configure this, create an elastic IP for the Ektron EC2 instance. This enables Solr's asset transfer client to connect to Ektron's asset transfer service.

  1. From the Amazon Web services Ektron Windows Service uses Windows Web Services to perform these activities: • Schedules the future publication and removal of content. • Transmits notifications that a sync should be started between the staging and production servers. • In the 3-Tier feature, transfers data between servers. console, go to EC2 > Network & Security > Elastic IPs.
  2. Select the Ektron machine name and click Allocate New Address.

  3. The Associated Address dialog appears. Check the Ektron server that will work with this Amazon Solr instance.
  4. On the Solr VM, open the hosts file (C:\Windows\System32\Drivers\etc\hosts).
  5. Add a new host file entry. Insert the Elastic IP address of and the name of the Ektron EC2 instance.

Store resource files in the Amazon S3

Follow these steps to allow Solr search to crawl Ektron resource files stored in the Amazon Simple Storage Service (S3). To set up this capability, update the following section of your site's siteroot\web.config file, in the <appsettings> section.

<add key="ek_CloudStorageType" value="" />
<add key="ek_CloudAccountId" value="" /> 					
<add key="ek_CloudAccountKey" value="" />
<add key="ek_CloudContainer" value="" />
  • ek_CloudStorageType—Enter AMAZON.
  • ek_CloudAccountId—Enter your Amazon account access key.
  • ek_CloudAccountKey—Enter your Amazon account secret key.
  • ek_CloudContainer—Enter the S3 bucket name.

For these changes to take effect:

  1. On the Ektron EC2 instance, restart the EktronWindowsServices40 and EktronAssetTransferServer.
  2. On the Solr EC2 instance, restart EktronAssetTransferClient2.0 and EktronSolrAdminService.
  3. Register the site again. See Part 9: Register the load-balanced site
  4. All new resource files will be stored in the S3 bucket. Verify that they are searchable from the Ektron site.

Setting up Solr Search in the Azure cloud

If you want to deploy an Ektron website to the Azure cloud, deploy Solr search to Azure cloud, then connect both cloud-based systems, you have 2 options:

Pushing an Ektron site to the Azure cloud and connect it to Solr search

Follow this procedure of you want to install an Ektron site to an on-premises server then push the site up to an Azure VM. Next, deploy Solr to the cloud. Finally, connect both cloud-based systems.

Part 1: Deploy Ektron Site to the Cloud

Follow the steps in Using Ektron in the Windows Azure Cloud to deploy an Ektron site to the Azure cloud. But note this important difference:

You cannot update the ektron.cms.framework.unity.config file directly on the Ektron VM that hosts the Solr search provider. This restriction occurs because, when the machine reboots, your changes are lost. So, follow these steps to update the ektron.cms.framework.unity.config file.

  1. During the Ektron installation, select Solr. If you did not do this, see Changing your search provider to Solr.
  2. Copy the siteroot/ektron.cms.framework.unity.config to: 
    C:\Program Files (x86)\Ektron\CMS400vxx\CommonFiles\Configs
  3. Run the initial cloud deployment.

If you need to make these changes after the initial cloud deployment, update the unity.config file in the C:\sync\azuredeploy\GUID\HostedServiceName\’HostedServiceName’_web folder.

To upload this file to a cloud-based site, use the EktronConfigUploader.exe utility in C:\Program Files(x86)\Ektron\EktronCloud\bin\ms\2.0.0.0 .

Part 2: Create Azure Solr search VM

Create an Azure virtual machine to host Solr search.

  1. Log into https://manage.windowsazure.com.
  2. Click Virtual Machines > New.

  3. Click Quick Create.
  4. For the new server, enter a DNS Name.
  5. For the image, choose Windows Server 2012.
  6. Choose a large size with at least 8 GB of memory.
  7. Enter your user name and password.
  8. At Region/Affinity Group, select the region selected when setting up the Ektron site (in Using Ektron in the Amazon EC2 Cloud).
  9. Click Create a Virtual Machine.
  10. Wait until the status changes from Starting (Provisioning) to Running.
  11. Select the VM and click Connect.

  12. The following dialog appears. Click OK to connect to the virtual machine.

  13. When you are prompted for Windows credentials, prefix your username with the DNS Name you created in Step 4.

Part 3: Set up Solr search on Azure VM

This section explains how to set up Solr search on the Azure VM.

  1. Apply all prerequisites to the Azure VM. You already set up an Ektron site in Using Ektron in the Windows Azure Cloud (PaaS).
    • An Azure database server is required. You have 2 options:
      • Use the database server that hosts Ektron.
      • Create a new database server.
    • If you installed Ektron manually on an Azure VM (see Installing Ektron site on Azure cloud and connecting it to Solr search), follow these steps to enable the crawling of assets and private assets.
      1. On the virtual machine that hosts the Ektron site, locate the Public Virtual IP (VIP) address.

      2. On the virtual machine that hosts Solr search, open the hosts file (located in %systemroot%\system32\drivers\etc).
      3. Into the hosts file, insert the VIP obtained in Step 1, followed by the hostname of the VM that hosts Ektron.

  2. On the Azure VM, copy the following into a .bat file and run it. This code fixes a problem in which Java JDBC to SQL Azure connections are being forcibly closed.
    REG ADD HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters
 /v KeepAliveTime /t REG_DWORD /d 30000
    REG ADD HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters 
/v KeepAliveInterval /t REG_DWORD /d 1000
    REG ADD HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters 
/v TcpMaxDataRetransmission /t REG_DWORD /d 10
  3. On the Azure VM, register ASP.NET.
    1. As an administrator, open a command prompt.
    2. Enter cd C:\Windows\Microsoft.NET\Framework64\v4.0.30319
    3. Run C:\Windows\Microsoft.NET\Framework64\v4.0.30319>aspnet_regiis.exe -i
  4. Copy your storage access account and primary key to the Azure web.config file. This enables the Asset Transfer service to correctly transfer Azure assets to Solr Search VM. To do this:
    1. On the Azure VM, find your storage account primary key by navigating to Storage Accounts > your storage account.

      NOTE: You created the storage account when you deployed an Ektron site to the Azure cloud, as described in Deploying a Site Package to the Azure Cloud.

    2. Click Manage Access Keys.
    3. Copy the Storage Account Name and Primary Access Key. you will paste them in Step g.
    4. Log on to the on-premises Ektron server.
    5. Open the file C:\sync\azuredeploy\GUID\HostedServiceName\HostedServiceName_web\web.config.
    6. Within the <appsettings> tag, add these keys. Insert the values you collected in Step 4c.
      <add key="ek_CloudStorageType" value="AZURE"/>
<add key="ek_CloudAccountId" value="storage account name"/> 
<add key="ek_CloudAccountKey" value="storage account primary key"/>
    7. Restart the Azure VM.
  5. Push the web.config changes to your Azure site. To do this:
    1. On the on-premises Ektron server, go to C:\Program Files (x86)\Ektron\EktronCloud\bin\ms\2.0.0.0.
    2. Open EktronConfigUploader.exe.
    3. Enter your storage account name and storage account primary key. (You obtained them in Step 4.)
    4. Click Browse and go to the web.config file that you updated in Step 4.
    5. Click Upload.
  6. Reboot the Worker Hosted service. This, in turn, restarts the Ektron Windows service. To do this:
    1. Log into https://manage.windowsazure.com.
    2. Click Cloud Services.
    3. Click your cloud service. (You created the cloud service when setting up the Ektron site in the Azure cloud.
    4. Click Instances.
    5. Click the instance hosted service name_worker_IN_0.
    6. Click Restart.
  7. Log into the VM you created in Part 2: Create a security group.

Part 4: Run the Solr Installation

  1. Contact your Ektron account manager to download the Solr installation file.
  2. Log in as a Windows Administrator user.
  3. Download and run the Ektron Solr installation file as an administrator.
  4. Accept the license agreement.

  5. On the Deployment Type screen, choose Install Solr on a single server.

  6. Select a folder in which to save the Solr installation files.

  7. You are prompted to select a base port. After you select a port, it and 9 consecutive ports are designated for Solr use. For example, if you accept the default port 7600 as the base, ports 7600 through 7611 are used for Solr.

    NOTE: Do not use a port number below 1025. Also, do not use ports 6080, 6081, and 8732: they are reserved for other Ektron components.

  8. Enter credentials (domain, user name, and password) of the Windows user account under which Solr query services will run. This account manages Solr components and authenticates queries.

    The account must have permission to

    • write to the search 2.0 folder and its subfolders
    • start the Solr admin service
    • log on as a service. If the user does not have "Log on as a Service" right, you are prompted to assign it.

    If you are using a domain account, the Netlogon Service must be enabled and running.

    `

  9. Enter a fully qualified DNS name for the Azure SQL Server that will host the Solr configuration database. Then either select Trusted Connection or enter the username and password of the user who will create the search configuration database. If you enter a user, select someone who is either the server-level principal login or assigned the dbmanager database role. See also: Managing Databases and Logins in Windows Azure SQL Database.

  10. Enter a unique name for your search configuration database. Spaces are not allowed.

  11. Enter credentials for a SQL login account that Ektron search components will use to access the search configuration database.

  12. Enter the fully-qualified DNS name of the server on which you are running the Solr installation. The Ektron server must be able to access this host.

  13. Select a folder in which Solr will store its index data.

  14. Specify the location of your asset cache. The asset cache is used only if the Ektron asset files are not stored on your Solr search server.

  15. A summary of installation choices appears. If everything looks OK, click Install.

    Key points on this screen:

    • The query service port is always the base port.
    • The screen lists your search service account, configuration database, and the port number assigned to the following:
      • Query service. Outbound from Ektron server
      • Admin service. Outbound from Ektron server
      • Tomcat connector. For local communications between Solr and Tomcat
      • Tomcat shutdown. For local communications between Solr and Tomcat
      • ManifoldCF. For local communications between Solr and ManifoldCF
      • Asset Transfer Client. Two-way between Ektron and Solr servers

    IMPORTANT: The summary screen's first 2 ports have an asterisk (*). On the Ektronserver, adjust the firewall to allow these ports to communicate between your Ektron site and the Solr service.

Part 5: Add Inbound Rules and Endpoints

  • Add inbound rules to allow TCP communication for Solr Search components and the Solr Administration Service. You defined these ports while completing Part 4: Run the Solr Installation. To do this:

    NOTE: Leave firewall status "Turned On."

    1. Go to Server manager > Windows Firewall with Advanced settings > Inbound Rules > New Rule.
    2. On the Rule Type screen, choose Port.
    3. On the Protocols and Ports screen, choose TCP. In the Specific local ports field, enter your base port and base port + 1. For example, 7600-7601.
    4. Click Next then Allow the connection.
    5. On the When does this rule apply? screen, check all boxes.
    6. Assign a name to the rule.
    7. Click Finish.
  • Add 2 endpoint Part of a DXH connection, it is the combination of an endpoint (a URL or API path) and authentication credentials that allows communication between 2 software instances. You define DXH endpoints in your site’s web.config file.s. To do this:
    1. Return to the Windows Azure portal, https://manage.windowsazure.com.
    2. Click your virtual machine.
    3. Click Endpoints > Add.

    4. The following dialog appears. Click the -> (next) button.

    5. Add 2 endpoint Part of a DXH connection, it is the combination of an endpoint (a URL or API path) and authentication credentials that allows communication between 2 software instances. You define DXH endpoints in your site’s web.config file.s that make Ektron Solr components on the Azure Solr Search VM accessible from the Ektron VM. To do this.

      • Name: SolrSearchSite
        • Protocol: TCP
        • Public and private ports: base port (by default, this is 7600)

        NOTE: To identify the base port number, log on to the virtual machine, open IIS, click EktronSolrSearch2.0 > bindings, and view the port value.

      • Name: SolrAdminSvc
        • Protocol: TCP
        • Public and private ports: base port plus 1 (by default, this is 7601)
      • Do not check Create a Load-Balanced Set and Enable Direct Server Return boxes

    • Part 6: Test and register Solr on the Azure VM

    1. Verify you can reach the Solr Search components by using this ping Url under the EktronSolrSearch site.
      https://DNSname.cloudapp.net:baseportnumber/ping.html
      For example, if the DNS name you assigned in Part 2: Create a security group is solrsearch, and your base port is 7600, the URL would be https://solrsearch.cloudapp.net:7600/ping.html.
    2. Register site using this URL.
      https://DNSname.cloudapp.net:baseportnumber/SolrAdminConsole/index.html

      See also: Part 2: Add a storage account in Azure manager

    SQL Azure database firewall permissions

    • To understand server-level and database-level SQL Azure permissions, refer to this Azure documentation.
    • If your SQL Azure configuration > Windows Azure Services box is checked or the allow all azure IPs rule is added to your master database using exec sp_set_database_firewall_rule N'AllAzureIps','0.0.0.0','0.0.0.0', any database in this SQL Azure instance is accessible by any Azure IP.
    • To grant your Azure Search VM permission to access the search configuration database (created by the SolrInstall), add your Azure Search VM to your search configuration database's firewall rules, and remove server-level permissions.
      • Add the public virtual IP of your Azure Search VM to your database firewall rules (after connecting to your search config database). Example:
        exec sp_set_database_firewall_rule N'ALLAzureIPs','137.135.70.139','137.135.70.139'
      • To delete server-level permissions using the Azure portal:
        1. Select your SQL Azure server.
        2. Configure it.
        3. Remove rules, or revoke them in the master database, using
          exec sp_delete_database_firewall_rule N'ALLAzureIPs'

        Azure recommends waiting 5 minutes for rule changes to take effect.

    • If you have one IP address in your server-level rule and another IP address in your database-level rule, both the IPs have access to the search configuration database. Server-level rules apply to all databases in an instance, while database-level rules apply to specific databases.
    • To list all rules, use
      select * from sys.database_firewall_rules

    Installing Ektron site on Azure cloud and connecting it to Solr search

    Follow this procedure if you want to manually install an Ektron site on an Azure VM. Next, deploy Solr to the cloud. Finally, connect both cloud-based systems.

    Prerequisites

    • Contact your Ektron account manager to obtain a link to download EktronCloudSetup.exe.

    Part 1: Provision a virtual machine for the Ektron site

    1. Log into https://manage.windowsazure.com.
    2. Click Virtual Machines > New.

    3. Click Quick Create.
    4. For the new server, enter a DNS Name.
    5. For the image, choose Windows Server 2012.
    6. Choose a large size with at least 8 GB of memory.
    7. Enter your user name and password.
    8. At Region/Affinity Group, select the region selected when setting up the Ektron site (in Using Ektron in the Amazon EC2 Cloud).
    9. Click Create a Virtual Machine.
    10. Wait until the status changes from Starting (Provisioning) to Running.
    11. Select the VM and click Connect.

    12. The following dialog appears. Click OK to connect to the virtual machine.

    13. When you are prompted for Windows credentials, prefix your username with the DNS Name you created in Step 4.

    Part 2: Add a storage account in Azure manager

    You need a storage account to store site resources in the cloud storage service, and to store and synchronize asset An external file, such as a Microsoft Word document or image, stored in one of these Ektron siteroot folders: assets, privateassets, uploadedfiles and uploadedimages. An asset can be managed like native Ektron content.s, private assets, uploaded files, uploaded images and the asset library. To create a storage account:

    1. From the home page of https://manage.windowsazure.com, click Storage. The Storage screen appears.
    2. At the bottom of the screen, click New. The New panel opens.
    3. Choose Data Service > Storage > Quick Create.
      1. Specify a URL Universal Resource Locator name for your cloud service (for example, cloudservicename). The .core.windows.net will be appended to your name to complete the URL.
      2. Choose the Location or Affinity Group from the drop-down list. For example, East US.
      3. Leave Enable Geo-replication checked if you have multiple servers.

    4. Click Create Storage Account at the bottom of the panel.

    Part 3: Install Ektron site on virtual machine

    On the Azure VM, install Ektron. See Installing Ektron.

    Part 4: Run EktronCloudSetup.exe on virtual machine

    On the Azure VM, install EktronCloudSetup.exe.

    After you download and install EktronCloudSetup.exe, the Cloud Manager is added to the Workarea Settings.

    Part 5: Run security configurator

    1. On the Azure VM, click the Windows Start button > All programs > Ektron > CMS400 v9x > Utilities > Security Configurator. Right click and choose Run As Administrator.

      If you're using Windows 8 or 2012, press the Windows key ()/Q then enter Security Configurator. Right click and choose Run as Administrator.

      The security configurator screen appears.

    2. Check the Remote Server Certificate Folder radio button. Then use the browse button to go to the Ektron\EktronCloud\bin\worker\certificate folder.
    3. Click Apply.

    Part 6: Add inbound rules for ports

    On the Amazon Solr server, create inbound rules to allow connections to Ektron port 8732 and database port (for example 1433). The rule enables the Ektron VM to communicate with the Solr Admin Console. To do this:
    1. On the Amazon Solr server, go to Server Manager> Windows Firewall with Advanced settings > Inbound Rules > New Rule.

      NOTE: Leave firewall status "Turned On".

    2. On the Rule Type screen, choose Port.
    3. On the Protocols and Ports screen, choose TCP. In the Specific local ports field, enter 8732.
    4. Click Next then Allow the connection.
    5. On the When does this rule apply? screen, check all boxes.
    6. Assign a name to the rule.
    7. Click Finish.
    8. Repeat steps 1-7 for the database port.

    Part 7: Add endpoints for port 8732 and the database port

    1. Return to the Windows Azure portal, https://manage.windowsazure.com.

    2. Click your virtual machine.

    3. Click Endpoints > Add.

    4. The following dialog appears. Click the -> (next) button.

    5. Add your Ektron Windows Service port.
      • Name: EWSPort
      • Protocol: TCP
      • Public and private ports: base port 8732
    6. Repeat steps 1-5 for the database port.

    Part 8: Update the Ektron web.config

    1. On the Azure VM, open the Ektron site's siteroot\web.config file.

    2. Replace the unity.storage section with this:

      <unity.storage>
<assembly name="Ektron.Storage" />
<assembly name="Ektron.Cloud" />
<namespace name="Ektron.Cloud.Azure.Storage" />
<namespace name="Ektron.Storage" />
  <container name="storageContainer">
    <register type="IFileService" mapTo="CloudFileService" />  
    <register type="IDirectoryService" mapTo="CloudDirectoryService" />  
  </container>
</unity.storage>

      Save but do not close the web.config file. You will update further soon.

    3. Obtain your Azure Storage Account Name and Primary Access Key. To do this:
      1. In a browser, go to https://manage.windowsazure.com.
      2. From the left panel, click Storage. The Storage screen appears.
      3. Select the storage account you created in Part 2: Add a storage account in Azure manager.
      4. Click Manage Access Keys at the bottom of the page.

      5. A screen shows your Storage Account Name and Primary Access Key.
    4. Within the web.config file, find the following element:
      <add key="StorageConnectionString" 
value="DefaultEndpointsProtocol=http;AccountName={0};
AccountKey={1}"/>
    5. Replace {0} with the Storage Account Name you obtained in Step 3. Then, replace {1}with the Primary Access Key that you obtained in Step 3.

    6. Within the web.config file, locate the <appsettings> section then add these keys within it.

      <add key="ek_CloudStorageType" value="AZURE" />
<add key="ek_CloudAccountId" value="MyStorageAccountName" />
<add key="ek_CloudAccountKey" value="MyKeyValue" />
<add key="ek_CloudContainer" value="" />
    7. In the above elements, replace MyStorageAccountName with the Storage Account Name you obtained in Step 3. Then, replace MyKeyValue with the Primary Access Key you obtained in Step 3.
    8. Save and close web.config.

    Part 9: Set up Solr search on the Azure cloud

    Follow these steps.

    Unregistering a Solr site

    If you remove an Ektron Web site which uses a Solr search server, use this procedure to unregister the Ektron site from Solr. This is especially important if you will install another Ektron site in the same location as the one that was removed.

    1. On the server on which you completed Part 1: Run Solrinstall.exe, and using an HTML5 CSS3-compliant browser All browsers are supported EXCEPT for the following versions *and older*: IE 8, Chrome 3 and older, Firefox 3 and older, Safari 3 and older, and Opera 10 and older., click Windows Start menu > All Programs > Ektron > Ektron Solr Search 2.0 > Ektron Solr Administration. If you're using Windows 8 or 2012, press the Windows key ()/Q then enter Ektron Solr Administration. The site registration screen appears.
    2. Enter the host name of the Ektron server then click Connect.
    3. Click the Search System Management tab.
    4. Under Existing Configurations, find the Ektron database that was removed. If you are unsure, compare the code following the Ektron database name with the Search Configuration Details screen's Solr Core name field.

    5. Click the Remove button.

    Editing Solr registration information

    After a site is configured, the buttons below the database change to Edit and Start Over.

    Start Over is a full reconfiguration. The index, schema, and configuration data are completely overwritten, and the site’s data is re-indexed. Because the index and supporting configuration are overwritten, search functionality is unavailable while the site is being registered and re-indexed.

    Consider this option if

    Edit updates an existing search configuration's settings. The index, schema, and configuration data remain intact. When settings are saved, the site’s data is reindexed. Search functionality remains available throughout this process.

    Consider this option if you need to

    Uninstall Solr search provider from an on-premises (local) server

    The following scenarios describe uninstalling the Solr search provider.

    IMPORTANT: Before uninstalling Solr, back up related files that have been customized. For example, these Ektron Search2.0 directory files may have been customized.
    * solrconfig.xml
    * schema.xml
    * dictionary files
    * wddf types
    * .txt files

    Simultaneously uninstall Ektron and Solr search provider

    To uninstall Ektron and Solr at the same time:

    1. Back up any Solr files that have been customized.
    2. Use the uninstall procedure described in Uninstalling Ektron.
    3. On the Uninstall dialog, check the Remove Ektron Solr Files and Services box.

    Uninstall Ektron, but do not uninstall Solr search provider

    To uninstall Ektron but leave Solr installed, use the uninstall procedure described in Uninstalling Ektron. On the Uninstall dialog, do not check the Remove Ektron Solr Files and Services box.

    Uninstall only the Solr search provider

    To uninstall the Solr search provider:

    1. Back up any Solr files that have been customized.
    2. Close the four command windows that open at the end of the Solr installation.
    3. Run the Ektron Solr installation file as an administrator. (You used this file to install Solr.)
    4. On the screen that has Upgrade and Remove options, choose Remove.

    Changing your search provider to Solr

    If you selected a search provider other than Solr during site setup and want to change to Solr, follow these steps.

    1. Uninstall Microsoft Search Server.
    2. On the Ektron server, open c:\Program Files (x86)\CMSvXXX\Utilities\SetSearchProvider.
    3. Click SetSearchProvider.exe.
    4. Select the website that you want to change.
    5. Select Apache Solr 4.0.
    6. Install the Solr search provider. See Installing Solr on a single on-premises (local) server.
    1. On the Ektron server, open c:\Program Files (x86)\CMSvXXX\Utilities\SetSearchProvider.
    2. Click SetSearchProvider.exe.
    3. Select the website that you want to change.
    4. Select Apache Solr 4.0.
    5. Install the Solr search provider. See Installing Solr on a single on-premises (local) server.
    6. On your search server machine, stop the Search Server Service.
    7. On your search server, open C:\Windows\System32\SiteConfiguration.exe and remove your website's entry.
    8. Run an IISReset.
    9. Start the Search Server service.

    Monitoring the status of search

    The Ektron Workarea provides 2 screens to help you monitor the status of your search. For information about managing the search crawl, see Managing the search crawl.

    Solr search configuration details screen

    The Solr Search Configuration Details screen provides information about

    To access the screen, go to the Ektron Workarea > Settings > Configuration > Search > Configuration.

    Solr Search node status screen

    The Solr Node Status screen provides information about the search crawl on 2 tabs.

    • Crawl status tab shows
      • The crawl status, as returned by ManifoldCF
      • The current and next indexing activities
      • For the most recent crawl: start time, end time, duration
      • If an incremental crawl request is pending See also: Full vs. incremental crawls.

    • Process Status tab shows the status of the (Ektron search) Java processes on the Solr Search server, such as Apache Tomcat, crawl database, and Apache ManifoldCF.

    From this screen, you can launch an incremental or full crawl using toolbar buttons. See also:  Full vs. incremental crawls.

    Apache Manifold CF status screen

    To view the Apache Manifold CF status screen, on the Solr server, click the Windows Start button > All Programs > Ektron > Ektron Solr Search 2.0 > Tools > Apache Manifold CF. If you're using Windows 8 or 2012, press the Windows key () /Q then enter Apache Manifold CF.

    The Apache Manifold CF screen displays the status of the jobs crawling your Ektron content. For more information, see ManifoldCF-End-user Documentation.

    Apache Solr console

    To view the Apache Solr console screen, on the Solr server, click the Windows Start button > All Programs > Ektron > Ektron Solr Search 2.0 > Tools > Apache Solr. If you're using Windows 8 or 2012, press the Windows key () /Q then enter Apache Solr.

    The Apache Solr console screen provides information about available logging levels, how to get management information about each core, the Java information about each core, and so on.

    For more information, see Using the Solr Administration User Interface.

    Managing Solr administrative tasks

    The following topics explain how to perform administrative tasks that are unique to Solr. The following list shows other search-related functions that apply to all search providers.

    Solr language support

    The following languages are supported, along with their two-letter language codes, without customization. Support is limited to Solr's linguistic capabilities (tokenizing, stemming, and so on).

    • English (en)
    • German (de)
    • Greek ( el)
    • Spanish (es)
    • French (fr)
    • Hindi (hi)
    • Italian ( it)
    • Japanese ( ja)
    • Korean (ko)
    • Dutch (nl)
    • Norwegian ( no)
    • Portugese (pt)
    • Russian (ru)
    • Thai (th)
    • Chinese ( zh)

    You may customize Solr to support additional languages. Refer to the guidelines specified <SolrInstallPath>\Search2.0\Solr\core0\conf\schema.xml.

    Managing the stopwords list

    When processing a search query, Ektron's Solr search ignores all words in the stopwords file for the selected language. The stopwords list's default location is: C:\Program Files (x86)\Ektron\Search2.0\Solr\cores\<core_id_number>\conf\<lang>\stopwords_<languagecode>.txt.

    The language code is a 2-character alphabetic code. For example, English is en.

    Your Solr core name appears on the Solr search configuration details screen.

    To modify the stopwords list, add or remove terms and save.

    NOTE: To change the path to the stopwords file, customize the schema.

    Filtering results with Solr refinements

    Refinements lets users explore search results by applying one or more filters. Here is an example from epicurious.com. Refinements are often called facets.

    When describing the refinements feature within Ektron's deployment of Solr search, the following terms are used.

    • Facet. A categorization of search results by values of an index field. A facet is made up of buckets: each bucket contains a collection of values, and the number of result items that contain that value.
    • Refinement Specification. A data structure defining a request for a particular facet to be returned with the results for the associated query. The specification may include data (thresholds) defining how the data should be categorized.
    • Refinement. The classification by which you want to restrict a search query. This can be thought of as an identifier representing a particular facet bucket. By applying a refiner to a query, you indicate that you only want to see results within that facet bucket.

    You can create refinements from any indexed content property, Smart Form field, or content metadata. Content properties are always indexed for searchable content. Smart Form fields and metadata that you feel may enhance your site’s search experience can be marked as searchable, at your discretion. To indicate that Smart Form fields are searchable, check the Indexed box on the properties dialog. For metadata, choose searchable type.

    Use the following fields on the Solr Registration screen to configure certain elements of working with facets.

    • Minimum Facet Count
    • Facet Sorting

    You request facets by adding refinement specifications to search criteria. A specification primarily identifies the target field for which a facet should be generated. For example:

    criteria.Refinement.Add(new IntegerRefinementSpecification(SearchContentProperty.Size));

    If you are working with index fields whose data types may represent ranged values, you may specify a collection of bounds to help categorize the data.

    To access the API for managing refinements:

    1. Install the Ektron 9.10 SP3 version of the OnTrek Sample site.
    2. From the site's home page, go to Developer Reference > Framework API > Ektron.Cms.Framework.Search.
    3. Look at KeywordSearchCriteria and terms with “refinement” or “facet.”

    NOTE: The Ektron Search API supports Solr field types that translate directly to string, long, double, and DateTime. Other field types (for example, LatLonType) are not supported as facets.

    NOTE: If you do not apply thresholds when submitting a refinement specification for a field with a ranged-value data type, data is categorized by each individual value. (Ranges are not auto-generated.)

    Every facet bucket references a refinement. The refinement represents a filter that, when applied to a subsequent query, ensures the results only include items corresponding to that facet bucket.

    Best Practices for Working with Refinements

    When you work with Solr refinements, consider the following advice.

    • Limit the number of refinement fields applied to a query. Six or fewer is a "good rule of thumb." More than that can be confusing and overwhelming, which undermines your goal of creating a better experience.
    • Use fields with a smaller set of unique values.
    • As you organize content and design Smart Forms, think about key properties that will enable users to identify content of interest.
    • Consider the significance of the property in determining the placement of a facet. For televisions, brand and price are probably the most significant properties, so should receive more prominent placement. Less significant properties do not require faceting and can be displayed within the product listing.
    • Tag content. Content tags are a quick and easy way to provide free-form classifications of your content and be a very effective facet. See also: Tagging content, library items, users, and groups with keywords.

    Managing Solr autoComplete

    Solr's Autocomplete feature suggests terms based on the user's input. For example, if you enter ba, autocomplete provides the following suggestions.

    If a user selects a word from the list, a query is executed using that word.

    Since the suggestion list is based on user input, it changes as a user enters more characters. By default, the suggestion list is based on words within searchable Ektron content. If you prefer, you can create a custom autocomplete suggestions list. You might do this to control which terms appear on the list, thereby directing site visitors to desired content. If you create a custom list, only its terms appear as suggestions—Ektron content is ignored. See also: Creating a custom autocomplete suggestions list.

    Additional points about working with AutoComplete:

    • AutoComplete works with single words only—if the user enters a space, AutoComplete terminates.
    • Suggestion list terms may include a special character, like power-shot
    • Special characters (such as exclamation point (!) or period (.)) within a search term are ignored.

      Apache Lucene supports the escaping of special characters that are part of the query syntax. Here is the current special characters list:
      + - && || ! ( ) { } [ ] ^ " ~ * ? : \

      NOTE: The period (.) is not listed a special character at Apache Lucene - Query Parser Syntax.

    Enabling autocomplete

    ...via Server Control

    To enable autocomplete for a templated server control, modify the SiteSearchController control by adding the following property: EnableAutoComplete="true". See also: SiteSearch templated server control.

    When using Solr search with templated controls, note these restrictions:

    • You cannot retrieve custom content.
    • The FacetsView control does not support ranged facets for integer, decimal, and date properties.

    ...via API

    For a sample of enabling autocomplete via the API:

    1. Install the latest version of Ektron's OnTrek Developer Sample site.
    2. From the site's home page, go to Developer Reference > Framework API > Ektron.Cms.Framework.Search > QueryPropositionManager.

    On an Ektron server, the page's path is siteroot\developer\Framework\Search\QueryPropositionManager.

    Creating a custom autocomplete suggestions list

    Follow these steps to create a custom autocomplete suggestions list.

    Part 1: Changing the source of autocomplete suggestions

    1. Access the Solr registration screen by inserting the following URL into a browser: 
      https://localhost:base port/SolrAdminConsole/index.html

      For example, if you selected the default base port of 7600 during the Solr installation, enter https://localhost:7600/SolrAdminConsole/index.html.

    2. Under the database that you are working with, click Show Advanced Options.
    3. Within the screen section Query Completion Dictionaries, check the box next to the default dictionary (term list).
    4. Click Register.
    5. After a few minutes, the change is registered. If the registration is successful, the change is registered in the Search configuration database.

    Part 2: Create a custom suggestions list

    Prerequisites

    • Your Solr Server and Core Name, available on the Workarea > Settings > Configuration > Search > Configuration screen.
    1. On the Solr search server, open the Ektron Search2.0 folder. (You selected its location when installing Solr. By default, it is installed to C:\Program Files(x86)\Ektron\ )
    2. Within that folder, open Solr\cores\core number\conf\staticfile.txt.
    3. Insert custom words into that file, each on a separate line, and save it.
    4. Access the Solr Administrator page: https://Solr server from Search Status screen/#/~cores/Solrcore name from Search Status screen. For example: https://localhost:7602/solr/#/~cores/56428657.
    5. Select Core Admin > your core > Reload.

    For more information on using the Custom Suggestions List, see https://wiki.apache.org/solr/Suggester > Dictionary.

    Maximum size of Solr assets

    For optimal performance, Ektron imposes a 10 MB limit on assets. Assets greater than 10 MB can be found by searching for their title or description.

    NOTE: The description is searchable via the API's SearchContentProperty.description property.

    Accessing Solr API documentation

    To access documentation for the Solr search API:

    1. Install the current version of Ektron's OnTrek Developer Sample site.
    2. From the site's home page, go to Developer Reference > Framework API > Ektron.Cms.Framework.Search.

    3. The search API classes, methods and properties are available for all search providers, including Solr. Tabs explain behavior or functionality differences for each search provider.

    Search Framework API Tutorial is an excellent resource for learning Ektron's Search APIs.

    Troubleshooting Solr

    After completing the Solr registration screen, you see this error:

    Validation of your input parameters has failed
    A request to Manifold CF failed unexpectedly

    Cause: The Search Process services stopped running.

    Solution: Run Process Manager as an administrator (C:\Program Files (x86)\Ektron\Search2.0\ProcessManager\ProcessManager.exe).

    HTTP Error 500.19 - Internal Server Error

    The requested page cannot be accessed because the related configuration data for the page is invalid.

    Cause: The modules section is locked at the server level.

    Solution: IIS > Configuration Editor > System.webServer/modules > Unlock section.

    No search results in the Workarea. You see the following error in the Windows Event Viewer/WindowsLogs/Application:

    Timestamp: 12/16/2013 10:06:32 PM
Message:Ektron.Cms.Search.SearchException: An error occurred while submitting the query.
---> System.ServiceModel.ProtocolException: The remote server returned an unexpected
response: (405) Method Not Allowed.

    Cause: WCF Activation is not enabled.

    Solution: For IIS 7.5, use the Add Features Wizard. Go to .NET Framework 3.5.1 Features > select both WCF Activation and HTTP activation .

    For IIS 8 (Windows Server 2012), use the Add Roles and Features Wizard. The HTTP Activation feature is located under .NET Framework 4.5 Features > WCF Services.

    https://www.iis.net/learn/web-hosting/configuring-servers-in-the-windows-web-platform/windows-communication-framework-wcf

    Known Solr issues and limitations

    Installation and launch

    • Whenever you use Solr, log in as the user who installed it. Do not let that user's credentials expire.
    • If you need change port numbers selected during installation, you must reinstall Solr.
    • If you upgrade your JDK version, you must replace it in the same directory to which you originally installed. You may not install more than one JDK version to that directory.

    Site registration

    • While re-registering a site, there may be an error (returned by ManifoldCF) while deleting a pre-existing connection corresponding to your core.

      Error description

      During site registration, old jobs for existing connectors (output and Ektron connector) are deleted prior to deleting the connectors themselves. Although the jobs are deleted, Manifold may perform clean-up operations on deleted jobs. During clean up, attempts to delete connectors for such jobs may fail.

      In this case, the site registration process attempts to delete the connection again after a specific interval. You can configure the maximum number of re-tries or the retry interval in the Ektron Solr Admin Service config file (C:\Program Files (x86)\Ektron\Search2.0\Administrative Service\ Ektron.Cms.Search.Solr.ServiceHost.exe.config). Within that file, edit the MaxDeleteAttempts and MaxDeleteInterval properties. Then, restart the Ektron Solr Admin Service.

      Solution

      If you see errors like Exceeded MaxTries while trying to delete output/repository connection in the Ektron Solr Admin Service logs (Verbose logging), increase these parameters or wait for the jobs to be completely removed from Manifold CF. Depending on the size of your index, the clean up of a deleted job may take a few minutes.

    • After installing Solr, until you register at least one site, if you attempt to browse to the Solr Admin screen, this message appears: There are no Solr cores running. Using the Solr Admin UI currently requires at least one Solr core.

      After you register a site, this message disappears.

    • If you edit the schema.xml file and try to re-register your site, an error is thrown.

      Error Description

      The site registration process encountered an error while attempting to remove 
an existing Solr core for your site. Could not find a part of the path 
'C:\Program Files (x86)\Ektron\Search2.0\Solr\cores\1ca92f29\conf.

      Solution

      Stop and start the Manifold Agent and Tomcat.

    • If you reload a core and try to register your site again, an error is thrown.

      Error Description

      The site registration process encountered an error while attempting to remove 
an existing Solr core for your site. The process cannot access the file 
'tlog.0000000000000000000' because it is being used by another process.

      Solution

      Stop and start Tomcat.

    General issues

    • If you used the Framework API and templated controls to create search controls using Microsoft Search Server, you can reuse that code with Solr for the most part. The Ektron Developer Reference documents functional differences among search providers.
    • Only templated server controls and the Framework API are supported.
      • Since version 8.5, the following controls and API are not supported:
        • IndexSearch server control
        • Web Search server control
        • ProductSearch server control
        • CommunitySearch server control
        • Search APIs that predate the Framework API > Ektron.Cms.Framework.Search
    • To improve performance, assign to the user account under which Solr services will run permission to lock pages in memory.

    Crawling

    • If a full crawl fails, the next crawl is a full one (even if a request for an incremental crawl was received).

    Search results

    • The search ignores words in a custom summary.
    • If you search a Web calendar title, the results include random words with the search text.
    • Some results from PageBuilder pages return concatenated text.
    • In some asset summaries, asterisks (*), question marks (�), and other special characters appear.

    • If you

      1. Import a .doc file to the Ektron Workarea.
      2. Copy and paste that file in another folder.
      3. Run a crawl.
      4. The second document can only be searched by its title.
    • If you
      1. Publish a content block.
      2. Crawl the content.
      3. Edit the content block, and set the golive date to a future date.
      4. Publish the content.
      5. Crawl the content, you see this error:
        Error: java.lang.IllegalArgumentException: Source input stream may not be null
    • If you
      1. Install and register a site.
      2. Open the Solr admin console and reload your core.
      3. Try to register the site again, you see this error:

        The site registration process encountered an error while attempting to remove an existing Solr core for your site.

        The process cannot access the file 'tlog.0000000000000000000' because it is being used by another process.

      4. To resolve, stop and start Tomcat to register the site again.
    • Solr misinterprets some complex queries.
    • The Workarea’s search results include content for which the user does not have read permission. If a user clicks such content, he is denied access.
    • If a search term exists in a document in a location beyond 100,000 characters, the document appears in search results but its highlighted summary is blank.

      If desired, you can increase the number of characters beyond which the highlighted summary appears with search results. If you do, you may see a degradation in your search's performance. To increase the number of characters, follow these steps.

      1. Open C:\Program Files (x86)\Ektron\Search2.0\Solr\cores\[CoreName]\conf\solrconfig.xml.
      2. Increase the value of the h1.maxAnalyzedChars property.
        <int name="hl.maxAnalyzedChars">100000</int>

    Solr ManifoldCF database maintenance

    See https://portal.ektron.com/KB/10171/.

    Templated search controls

    The FacetsView control does not support ranged facets for integer, decimal, and date properties.

    Open topic with navigation